The Social Coding Contract
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 65 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/37648 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Coding theoryDesign by contractExecution unitUsabilityOpen sourceLine (geometry)TwitterForm (programming)Open sourceComa BerenicesSoftware testingObject (grammar)Goodness of fitContext awarenessPhysical systemPatch (Unix)BuildingCodeSoftware maintenanceUniverse (mathematics)Open setFreewareLibrary (computing)Multiplication signComputer animation
02:13
Open sourceHand fanBitTerm (mathematics)Radio-frequency identificationLattice (order)WordPay televisionGroup actionOpen sourceOpen setStructural loadCodeHand fanSpacetimeRight angleComputer animationLecture/Conference
03:18
Survival analysisArithmetic progressionGroup actionSoftware developerGoodness of fitGraphics tabletInternetworkingComputer animationLecture/Conference
03:52
InternetworkingArithmetic progressionDistribution (mathematics)Core dumpLecture/ConferenceComputer animation
04:30
Maxima and minimaArithmetic progressionPoint (geometry)Natural numberMultiplication signRight angleWave packetLevel (video gaming)Ocean currentComputer animationLecture/ConferenceMeeting/Interview
05:06
Open setOpen sourceComputer fileOpen sourceInternetworkingMultiplication signArithmetic progressionSpeech synthesisComputer fileOpen setSoftware maintenanceCartesian coordinate systemRevision controlPhysical systemMereologyBuildingEngineering drawingProgram flowchart
05:44
Physical systemIndependence (probability theory)Cartesian coordinate systemCodeLogicSocial classProcess (computing)Java appletComputer fileWebsiteBytecodeGroup actionFormal languageLibrary (computing)Stack (abstract data type)Engineering drawingProgram flowchart
06:44
Limit (category theory)Group actionComputer fileGraph (mathematics)Image resolutionRevision controlLibrary (computing)Process (computing)Multiplication signMultiplicationRun time (program lifecycle phase)Limit (category theory)Maxima and minimaArithmetic progressionPoint (geometry)WindowTerm (mathematics)UMLComputer animation
07:56
Installation artCloud computingPoint (geometry)FamilyGoodness of fitCountingCartesian coordinate systemProduct (business)Slide ruleBlogInformationRevision controlMobile appMultiplication signImage resolutionProjective planeCodeLatent heatComputer animationMeeting/Interview
09:29
Revision controlInstallation artFiber bundleDirected setGroup actionImage resolutionRevision controlProjective planeDirection (geometry)Information1 (number)Fiber bundleHand fanRight anglePerturbation theory
10:03
Revision controlPoint (geometry)Image resolutionGame controllerMathematicsSoftwareWordEndliche ModelltheorieLibrary (computing)Domain nameModule (mathematics)TriangleSoftware maintenanceSquare numberInformation securityObject (grammar)BuildingVideoconferencingAuthorizationMultiplication signLatent heatProjective planeComplex numberControl flow
12:02
Kolmogorov complexityCodeCartesian coordinate systemProjective planeGroup actionGoodness of fitInstallation artConfidence intervalSoftware developerVideo gameMaterialization (paranormal)Complex (psychology)Arithmetic meanOpen sourceBitComputer fileComputer animation
13:03
Open sourceSoftware maintenanceBitOpen sourceSoftware maintenanceHacker (term)Software developerLibrary (computing)Musical ensembleRight angleGoogolShared memoryComputer animation
14:12
Software maintenanceProgrammable read-only memoryBuildingFreewareCASE <Informatik>MassData conversionPoint (geometry)Duality (mathematics)Software maintenanceDesign by contractWellenwiderstand <Strömungsmechanik>Library (computing)Projective planeCodeRevision controlRoundness (object)Limit (category theory)Fitness functionEnterprise architectureException handlingOptical disc driveReal numberEmailDependent and independent variablesCommitment schemeGoodness of fitBitOpen sourcePerfect groupNeuroinformatikArithmetic meanMultiplication signInstance (computer science)Perspective (visual)Uniform resource locatorCategory of beingGame controllerDifferent (Kate Ryan album)Greatest elementTrailTerm (mathematics)Computer animation
19:33
Software maintenanceDifferent (Kate Ryan album)Category of beingSound effectTwitterRight angleTelecommunicationNatural numberSoftware maintenanceAsymmetryInternetworkingOpen sourceSoftware developerProjective planeComputer animation
20:20
Software maintenanceProjective planeOnline help1 (number)Point (geometry)Process (computing)Right angleArithmetic meanComputer animation
21:01
QuantumExpressionTablet computerConvex hullMusical ensembleOnline helpInfinityModule (mathematics)Software maintenanceCartesian coordinate systemRight angleOnline helpSystem callGroup actionEmailProjective planeService (economics)AuthorizationType theoryLatent heatPoint (geometry)Computer animation
22:11
Process (computing)Repository (publishing)Physical lawCartesian coordinate systemWave packetMultiplication signTwitterMetropolitan area networkElectronic mailing listAuthorizationComputer animationMeeting/Interview
22:53
Phase transitionBinary codeRun time (program lifecycle phase)Projective planeCartesian coordinate systemAuthorizationMetropolitan area networkCloud computingElectronic mailing listMultiplication signService (economics)Slide ruleComputer animation
23:47
IRIS-TCache (computing)BackupSlide ruleService (economics)1 (number)Arithmetic progressionQuicksortOpen sourcePoint (geometry)Offenes KommunikationssystemSingle-precision floating-point formatBackupDiscrete element methodMultiplication signCryptographyRight angleComputer animation
25:01
Point (geometry)Data managementProjective planeCodeSingle-precision floating-point formatMultiplication signService (economics)Analytic continuationTwitterGoodness of fitComputer animation
25:50
Point (geometry)Service (economics)Mechanism designSingle-precision floating-point formatOpen sourceCodePoint (geometry)Row (database)HookingOrder (biology)QuicksortGoodness of fitComputer animation
26:43
CloningFeedbackPhysical systemEmpennageMessage passingHacker (term)Web pageMessage passingMixed realityMultiplication signLine (geometry)Open sourceElectronic mailing listEmailWeb pageHard disk driveProjective planeHacker (term)Server (computing)Service (economics)Statement (computer science)Integrated development environmentProcess (computing)
27:22
Interior (topology)WebsiteStaff (military)Statement (computer science)Open sourceIntegrated development environmentMultiplication signProjective planeStandard deviationReal numberGroup actionDifferent (Kate Ryan album)GradientType theoryComputer animationLecture/Conference
28:06
Gamma functionQuantumGradientInformation securityOpen sourceCodeExploit (computer security)Installation artGradientProjective planeFlow separationCodeMultiplication signRevision controlInternet der DingeSingle-precision floating-point formatSoftware bugPlastikkarteComplex (psychology)Source codeOpen sourceInformation securityRight angleDisk read-and-write headExploit (computer security)Group actionCommitment schemeProper mapSoftwareCommutatorProcess (computing)Parameter (computer programming)Computer programmingPressureTerm (mathematics)Formal grammarAbsolute valueSemantics (computer science)QuicksortSoftware maintenancePoint (geometry)InternetworkingComputer animation
30:53
OvalString (computer science)Integrated development environmentVariable (mathematics)Function (mathematics)Parameter (computer programming)Open sourceState of matter2 (number)Line (geometry)Functional (mathematics)Variable (mathematics)ProgrammschleifeGroup actionSubject indexingVulnerability (computing)String (computer science)Multiplication signWebsiteCodeLoop (music)Computer animation
31:30
ProgrammschleifeString (computer science)Function (mathematics)Food energyComputer programCodeProgrammschleifeMultiplication signString (computer science)CodeBoolean algebraCognitionQuicksort2 (number)Operator (mathematics)Computer programmingFood energySoftwareFreewareComputer animation
32:19
Level (video gaming)Library (computing)1 (number)Source codePoint (geometry)CASE <Informatik>Coefficient of determinationComputer animation
32:57
Square numberMagnetic stripe cardOpen sourceOpen sourceNormal (geometry)Patch (Unix)Exploit (computer security)Lie groupGroup actionInformation securityShared memoryBoss CorporationMassVector potentialCASE <Informatik>Negative numberDependent and independent variablesComputer animation
33:46
TouchscreenOpen sourceModemSimulationTwitterEmailInteractive televisionOpen sourceFigurate numberVideoconferencingRight angleTouchscreenTelecommunicationMereologyForm (programming)Multiplication signPhysical lawOnline chatData conversionReal-time operating systemPhysical systemLecture/Conference
35:28
Strategy gameExtrapolationSystem programmingCodeLevel (video gaming)Video gameForm (programming)TelecommunicationGraph (mathematics)TouchscreenPhysical systemLevel (video gaming)Electric generatorStrategy gameOcean currentTwitterGroup actionRight angleProgrammer (hardware)Programming languageWritingCodePower (physics)Computer programmingCuboidComputer animation
36:49
1 (number)Physical systemLevel (video gaming)Multiplication signNatural numberOpen sourceHigh-level programming languageCodeSystems engineeringSystem callMixed realityComputer animation
37:39
System programmingPhysical systemPhysical systemMixed realityReal-time operating systemLevel (video gaming)CASE <Informatik>Row (database)Fault-tolerant systemTerm (mathematics)Line (geometry)Right angleComputer animation
38:39
IterationSystem programmingPlane (geometry)Control flowWeb browserCartesian coordinate systemNP-hardSystem callReal numberStaff (military)Physical systemPlanningLevel (video gaming)Spherical capInstance (computer science)Patch (Unix)Web browserVideo gameGame controllerLocal ringDifferent (Kate Ryan album)Right angleComputer configurationDependent and independent variablesMereologyDomain nameProcess (computing)Overhead (computing)Systems engineeringTime zonePerspective (visual)Term (mathematics)Semiconductor memoryScripting languageOrder (biology)Open sourceCycle (graph theory)Connected spacePoint (geometry)INTEGRALFrequency1 (number)Letterpress printingMultiplication signEntire functionBitJava appletBusiness objectMobile appBefehlsprozessorWeb applicationIterationGraph (mathematics)Computer animation
41:18
Perspective (visual)System programmingPerspective (visual)Reflection (mathematics)Physical systemLevel (video gaming)Product (business)Open sourceSoftware developerWeb-DesignerComputer animation
42:06
AreaOpen sourceMessage passingMereologyInformation securityRevision controlMultiplication signProcess (computing)Dependent and independent variablesCodeMobile appReal numberUltraviolet photoelectron spectroscopyComputer animation
43:20
Lattice (order)Software developerSoftware testingDoubling the cubeControl flowSelf-organizationLine (geometry)Multiplication signComputer animation
Transcript: English(auto-generated)
00:20
morning. Let's roll. My name is Justin. A fun fact, I get paid by the tweet. So if you follow me on Twitter and say hello, I'd love that or if you want to drop me a long form line, you can reach me hello at testdouble.com. Open source is good, right? Companies working with competitors, other companies on common tools and then turning around and sharing that for free.
00:42
Startups, now they can stand on the shoulders of giants and build great new things with just adding a little code on top, companies that couldn't exist otherwise. And then never before in the history of the universe has an individual that's not state sponsored or company sponsored been able to just do a little work of their own and then literally change how the world works.
01:02
But is open source good really? I mean companies love consuming open source but if you ever want to share an upstream patch much less open source a library, they suddenly are very stingy and skeptical of this open source thing. And then a lot of startups keep falling into the same trap of hoovering up all of this free stuff without understanding how it works and building maintainability nightmares right
01:23
as they get successful, they can't add new features anymore. And most of the maintainers I know are pretty burnt out, right? Like they don't like the fact that they're doing something for fun in their free time, companies are running on that stuff and then expecting customer support, you know, on nights and weekends. So today my goal is just to bring to light a handful of issues affecting open source.
01:45
And my only objective here is to encourage you to do the same thing because maybe if we can start to build a broader awareness of some of the systemic issues in open source, we can start to have ideas of how to fix them. And then maybe somebody will come along and start to create new creative solutions for those things and then we can, you know, start to live and realize the promise of
02:05
true openness, whatever that means. And then we're done. But again, today, very little minor, just looking at a handful of things. Topics such as dependencies, pulling back the curtain a little bit to show what it's
02:21
like to be a maintainer, issues of trust, adoption, security, and then some deep thoughts about how we interact with each other as humans as well as where I think the future is heading. I like to start off with term definition, the word ideology. Most of us think of the word ideology as like a political subscription or affiliation, what you believe, but I like this definition more.
02:41
They do not know it, but they are doing it. Ideology as the negative space that's driving our actions without us even realizing it. It's a quote from a dude named Karl Marx. Open source fans are a bunch of hippies, so I figured I'd start with the Marx quote. That comes from the book Capital, and Capital is an interesting book because as a work, it sits at the intersection between philosophy and economics.
03:03
I think it's an interesting subject to start with today because so does open source, right? We share all this code altruistically as if to earn karma from people we don't know, and yet there's all these companies out there making bucket loads of money off of open source, and every company that even doesn't contribute to open source needs it to get by. Thinking of capital and traditional economies, I want to chart the march of progress of
03:24
economics. In the beginning, everything was shitty. Everyone was just trying to get by, survival day to day. But then as groups of people started to form, specialization emerged. You know, you could go to one market for your veggies and another person for your meats, and through efficiencies, this opened the door to the development of human
03:42
culture. Recreation and art emerged. Industrialization further optimized this because now we could go to one place and get all kinds of goods. The internet totally inverted that, so now from my bed on my iPad, I can order things from anywhere in the world and have it all shipped to my door.
04:00
This is progress. But where does it lead us? You know, this year there were rumors that Amazon is actually using big data to predict what you're going to buy before you click one click, and they're actually shipping it to distribution centers near you in advance so that they can send it to you the same day or the next day. And some people are starting to ask questions like, is this march of progress actually
04:21
somehow taking away something that's been core to the human experience? So it's an unintended consequence. Another example of unintended consequences in the, like, progress of economics? Food, Inc., right? This documentary was a big hit, and look at that aggressive tagline, you'll never look at dinner the same way. But I think it would be more honest if that said, for like at least a month.
04:43
Because we can't change these. These are systemic issues. When you chart progress over time, there's a natural accretion of awfulness. It just mounts up until it gets to a point where we all freak out about it. And when we freak out about it, we think, we're going to fix this, right? But we can't just turn on a dime. Nothing stops this train. It's going to keep on getting worse before it gets better.
05:02
And maybe we can rein it into current panic levels later. Speaking of awfulness, let's chart the same march of progress about all of the tools that we use to suck in new dependencies from the open source world. So charting open source's progress over time, in the beginning there were just files
05:22
out there on the internet. And so if I wanted to build a system and pull in some open source, I'd have to go find it first, download it, and then literally check it into my version control, depend on it, and then logically it kind of continued on as part of my application. If it broke, I had to fix it. And that was a guard against pulling in too much because I didn't understand
05:41
it. And it was just more and more for me to maintain. Makefiles and common build system tools emerged as a great way to depend on stuff logically. So now I could build an application and depend on something like libxml, and it could be built on each of the systems that needed to be able to compile, but it existed as a separate entity. I could upgrade it separately, and I could view it as apart from my
06:04
application code. Java and its JAR files were another great innovation because now instead of having all these build systems configured appropriately, a single compiled bytecode could be distributed and then run anywhere. So with Java, I could literally go to a website, download a JAR, put it on my
06:21
class path, and it would work. And this was so convenient that it actually opened the door to those same sites saying, oh, and by the way, we depend on this third thing. That's what we call a transitive dependency. And what that allowed was really all of these libraries that we depend on to become small, more focused, and even, you know, when you think of transitive dependencies like Apache Commons emerged as almost an alternative
06:40
language stack within a language ecosystem. It was extremely novel. Ruby, you guys, have done great work making this even easier with Ruby gems and Bundler. Now, when I'm writing my gem file, I only say the things that I explicitly depend on, and those transitive dependencies are discovered for me, and their version resolution is handled for me automatically
07:00
without me even knowing. And I can have arbitrarily deep dependency graphs only really thinking about the stuff that I directly depend on. NPM has just taken this a step further because the Node.js runtime doesn't require, it allows you to load the same library multiple times in a single process, which means I declare my dependencies and then it just naively sucks up all the dependencies of my dependencies
07:23
and so on and so forth to these gigantic trees, and it gets really broad and really deep, up to the point where a very common thing for a Node.js library to run into is a support ticket saying, hey, I can't install this because it's literally longer than the Windows Max file path limit of 256 characters. Wish that was a joke.
07:42
We, this march of progress is optimizing for convenience, you know, getting somewhere more quickly. It's short-term progress and it's available to us for the low, low price of long-term fragility. The comedian Louis C.K. talked about this recently. It's true, everything that makes you happy is going to end
08:03
at some point, and nothing good ends well. It's like if you buy a puppy, you're bringing it home to your family saying, hey, look everyone, we're all going to cry soon. Look at what I brought home. I brought home us crying in a few years. Here we go. Countdown to sorrow with a puppy.
08:26
Our community's Louis C.K., a guy named Gary, he told me to build a small but non-trivial Rails app. An empty app will have 50 gems, but yours will end up with 75 to 100. Now go away for six months. Come back, update all your dependencies.
08:40
Your app no longer works. I know from experience in the Ruby community that this is true. But it's easy to start a Jekyll blog. It's easy to install Sass. It's easy to generate a Rails app. It's always easy right now, never in a year. The reason, I think, is that when somebody asks us what our application is,
09:00
we think of the code that we write as being our application. Even though, upon inspection, all of us would agree that our app is really the full stack of everything that we ship into production. It's never been easier to ship something to production, but the things that we're shipping to production have never been more complex. I'm guilty of this all the time. I say, oh, it's a Rails app, because it conveys a lot of information all at once.
09:21
I never think to say, oh, and Rails depends on Thor, this very specific version specifier. I don't even notice that. I didn't even know it until I made that slide. Even though 272 gems can no longer be installed in the same project because of version resolution conflicts. So Bundler, it hides some of this from us.
09:41
I think it could actually promote some of this information. It could tell us, hey, we just installed 10 direct dependencies and that conferred 43 transitive ones. Or it could say, the version specifiers on all those gems preclude the installation of 1,300 other gems out of our ecosystem of 88,000. It could tell us, hey, and by the way, if you were to run bundle update right now, it would be unable to update five gems to their latest version.
10:01
I'd love to know that proactively. Of course, if you're an NPM, a Node.js fan, you might point out, haha, version resolution doesn't affect NPM, and you're right, but there's other issues. I can depend directly. So let's pretend orange triangle is a dependency. I might directly depend on version C of it. One of my dependencies might depend on version B,
10:22
and one of its others might depend on version A, all at the same time. Now this is all well and good until I start to think about it. I think, well, what if I call my triangle and it gives me back a domain model object, at version C's library understanding, and I foolishly, admittedly, I pass that into my dependency, who passes that into version B of the same dependency.
10:42
As a library author, did I think about that when I was writing the library that that was possible? Probably not. So is it going to blow up, or is it going to work? The answer is nobody knows, and that's not great for understanding the software that we're building. Another issue is that as a library author, I can specify the exact version that I want of something,
11:01
but I can't control the specification underneath. So if orange triangle depends on a very loose specifier, like red square, for example, at version star, which means that the user's install time, they're just going to get the latest and greatest version, things could potentially break. In fact, on our project, LinemanJS, one day, we just started to get issues
11:21
that every new installation was failing, and we hadn't changed anything for like a month. What had happened was that one of our direct dependencies broke because it had too loose of a version specifier on something else, and so there was a breaking API change. And then to fix it, we realized that because the maintainer was out to lunch, we actually had to fork that dependency,
11:40
and then specify explicitly, I want version D, the last compatible version of this thing, and then push that up to NPM, push up a new version of Lineman to NPM, and now we're saddled with the maintenance and ownership of this NPM module that we don't understand, that we'll never look at, that if there's a security thing or whatever, it's just baggage now.
12:01
Here's a video that the team took of me that weekend. I feel like that a lot. So, we all understand that the code that we write for applications is the code that we need, and the code that we depend on directly is there for convenience to help us get started. And if we think about it, we can all agree
12:21
that the stuff that our dependencies depend on, that's, you know, brings some complexity, but when we think any deeper than that, it just starts to feel like risk. And how often do you ask yourself about your transitive dependencies, transitive dependencies? Not often, it's just anything deeper is mysterious. That's why I think a lot of people have been lampooning the phrase, full stack developer this year,
12:41
because nobody knows everything that they're shipping anymore. So, this makes me long for the good old days, sometimes, of, like, make files. Sure, they were painful, but maybe that was a healthy pain. You know, 30 years later, a lot of these old C projects still build correctly. Now, who's confident that when I run NPM install on my project 30 years from now, it's still going to work?
13:06
Let's talk a little bit about all these people ruining everything like me, open source maintainers, and what it's like to be one. I think the most important thing to know is that open source maintainers are not rock stars. They're just humans, and in fact, they're kind of just like extra early adopters.
13:21
The way I envision it is that a maintainer is like, on Google, looking for the thing that they need, and then realize, oh, it doesn't exist yet. I'll turn around and go make this thing, and work in the open and share it. An early adopter, and slightly less so, might just Google the same thing the next day, and then find it. And then their reaction might be, oh, sweet, I found this cool, brand new thing. I'm going to go share it on Hacker News and talk about it, because it's still hip and underground.
13:41
It's like finding a cool indie band that you want to share with people. And this starts putting the maintainer up on a pedestal, right, you know, promotes them, and they might get stars in their eyes, and get excited that people are using their thing, but because they're on that pedestal, other people on Hacker News don't mind at all pointing out, oh, it doesn't do X or Y or Z, and now the maintainer, he realizes their ego
14:00
is all wrapped up in the adoption of their tool, but because early adopters are usually just as competent as maintainers at building stuff, maybe they'll send in pull requests that fix those issues, and now the maintainer's really happy again. It's super up and down. The initial release of any new library, new open source stuff, is usually just scratching the itch of a person who has a need, so there's going to be rough edges,
14:20
and early adopters are great because they, as developers, often will send in pull requests or submit issues to help round things out, and what emerges is usually like something that's pretty ready to be consumed by the masses. Maybe you call that a version 1.0 candidate, or if you're, like, afraid of the implications of what 1.0 means in semantic versioning like I am, that's your version 0.84. So at this point, I'd love for the conversation to happen
14:45
between maintainers and early adopters. You know, sharing ownership, hey, let's own this thing together so that the early adopter can broaden the contribution base of this project. That typically doesn't happen. I'd even settle for just the maintainer saying, hey, let's make you a committer. That way you get notification of issues and stuff, and the early adopter agrees that they'll help sometimes.
15:02
But because those conversations don't happen at this point, the maintainer may as well be saying, hey, let's never communicate again. Sounds good, bye forever, and then they're off to the next thing because early adopters are always after the new shiny. So why don't maintainers just share control? Well, I think the reason is that they misjudge how much happiness this library is going to bring them because up until now, they got to scratch their own itch
15:21
and build a thing, and that was fun. And then, you know, they started to get acknowledgement for it, and that was great. Then they got to version 1, and now they can only imagine the sky's the limit. This thing's going to make me super happy. I'm going to be like DHH, and people are going to start holding conferences after me. Fear not because late adopters will disabuse them of this happiness. So from the maintainer's perspective,
15:41
if it scratched their itch initially, version 1 is probably mature enough to do what they need. So they might go a week without any commits or activity. They might go a month, and because they're also like early adopters, some new shiny thing's going to come along, and, you know, that might distract them and then whisk them away, and they're off into the next thing. A late adopter sees that,
16:00
and then their response is usually excitement to say, oh, look, no recent commits. This must be stable. Eight hundred stars. This sounds like a safe bet. Open source. That's free. That's good. And so there, you know, like very often, from years-old projects, I'll get an email being like, oh, we're so glad we found this. Thanks. So maintainer's needs start small,
16:20
but then we, you know, layer them on, round things out, and if we conceive of each library as a thing, and then we layer on top of it what a user needs, like what I need out of that library, it's never a perfect fit. There's going to be some things I wish that that library did that it doesn't do, and in that gap becomes a negotiation. I have to ask the question, well, do I submit a PR for this?
16:41
Do I work around it? Do I open an issue and ask them to implement a feature for me? And so a lot of those late adopters, like literally two days later, will come to the project, put a stern look on their face, and then start saying, wait a second, this thing doesn't enterprise my enterprise thing at all. How could they ignore such an obvious and important use case? And so this is where, like, entitled GitHub issues come from.
17:01
Missing obvious and important feature. Well, you know, our team thought this was a well-written and good library, except that it doesn't do this crazy nonsense. And so I try to be polite, and I respond like, I didn't know it had to do crazy nonsense. Could you please explain? But instead of a real reasoned conversation, all their coworkers just jump in and start plus one-ing it. And the implication is that I'm bad, and I should feel bad for not doing this stuff for them
17:22
and having anticipated their needs. So odds are, it's a weekend, I'm at the park with my wife, I'm having a beautiful day, and now I'm glued to my phone, getting really nervous and feeling inadequate. So I run home real quick, and I start working, hacking, hacking, hacking on the stuff, and I come back and I'm like, hey, guys, I just spent all weekend building this thing. Can somebody please just verify it works so I can close out the issue?
17:41
And now what happens at this step? I never hear from them again. And now I have a sad, because I just did all this work for free for strangers that they didn't appreciate, that they might not ever use, and now I'm saddled with it forever, and I'm gonna have to work around that little edge case in my code going forward and make the thing that I love less maintainable.
18:03
So this is how the, you know, projects tend to bring me less happiness as a maintainer over time, to the point where I hate most of my projects after they're three years old or more. So late adopters, you know, because these well-rounded version one things do the happy path well enough, late adopters typically are, when they're asking for something,
18:20
they're asking for more niche features than early adopters. This leads a lot of people in our community to assume that late adopters make better customers than users, and that's why you see things like donate buttons, people talking about dual licensing, GPL plus commercial, pro-feature add-on packs that they can pay for, maybe paid support contracts or consulting hours. All of this, I think, kind of misses the point,
18:41
because it's a motivation impedance mismatch here. People are building open source out of intrinsic motivation and drive that they have, and money isn't gonna solve that problem. And so if you want an open source library to do something and the other person doesn't, just throwing money at it isn't going to make them any more motivated to do it necessarily. I would much rather, if culturally,
19:01
we all got in the habit of letting maintainers feel free to say no. I feel like a jerk when I say, no, I can't do this feature for free for you. But even now, like when I ask for a pull request or something like that, what I really want to say often is like, I don't think that this is a good idea for the curation of this library long-term. I'm sorry.
19:20
And I feel like a total asshole when I say stuff like that. But I think that we'd be all a lot healthier if we were a little bit less entitled and when we opened an issue, we were more complimentary and asking for advice on how they might approach the same problem instead of assuming that the library needs more code in it. Trolls are a totally different category of people who make us unhappy. Obviously, they spit out hate, sometimes threats.
19:42
Whatever it is, we know that the effect that they have on other people is they make them want to give up and walk away. And so remember, maintainers, they're not rock stars. But because of the asymmetry, right, like a lot of famous maintainers have 15,000 followers on Twitter, we assume that they have their stuff together and we might even understand like, well, if they got that many followers,
20:00
there's probably a couple of trolls in the room, but it's not that big of a deal. But because of the asymmetric nature of communication, we don't see the fact if those trolls are the overwhelming majority of who's interacting with them on the internet. And they have an outsized impact on a lot of open source developer psyche. So when the maintainer quits, we're all like, whoa, that's weird. What happened? I didn't see any of that.
20:21
Seth Varga recently left the Chef project. He was a great contributor, did a lot of awesome stuff, but just a handful of trolls in the community were able to sap all the joy that he had from that project, further making us sad. And so maintainers at this point, this is where I often see the cries for help saying, hey, I'm burnt out. Can somebody please help me maintain this thing that has all these features that nobody wants anymore?
20:41
Hello, anybody? And at this point, no one's there, right? The early adopters who might have helped are long gone at this point, and this is exactly how a lot of projects stagnate and die. But even well-maintained projects up until, you know, by maintainers who do all the things right, no maintainer lasts forever.
21:01
Big invisible one this year was TJ Holloway. Chuck left Node.js for Go, and in the process, he handed off a lot of his, like, infinity billion NPM modules to other maintainers, but he actually sold Express.js to Strongloop even though he wasn't the primary maintainer on it anymore, and he didn't tell any of the maintainers. So it's really easy to screw this up.
21:21
Why? Of course, you know, beloved in our community, he left, and he had every right to close down all of his stuff and leave. And because he was highly visible, we all had forks of his stuff, and we were able to put it all back together again. But that's not true of most maintainers. So I like to think, like, what if there was an application for this that recognized that it's very human, very natural for people to leave and stop contributing at a certain point?
21:42
And I think it would be neat to be able to broaden the base of people who are contributing. So, like, what if I had a service where I could authorize RubyGems and NPM and GitHub, and it would, like, aggregate all of my projects, and I could actually, like, explicitly say, I need help on this one, and here's the type of help that I need. That way, when somebody goes and logs in, there's, like, a dashboard where they can see all the projects that they use that have asked for help in specific ways,
22:02
and then a call to action to say, yes, I'll offer to do that. That's a lot... It strikes me as way more successful than just sending off an email in the dark to a maintainer being like, I like your thing, can I help you? The reason that I did this talk today, the reason that I thought to put it together was because our friend and beloved colleague,
22:21
Jim Weirich, passed away. And when Jim died, I thought a lot of things, and it was very hard on me, but one of the things I thought about was, like, he's got this tremendous legacy, and I want to see if I can help. In particular, I have a favorite gem of his is rspec-given, and I wanted to see if I could go about helping adopt it, and that process was very complex and difficult, and it made me realize that when people
22:41
go to their lawyer to do estate planning, very few people ask, like, well, what's going to happen to my GitHub repositories when I die? And even if they had, the law hasn't really caught up to technology, and it never will, but maybe we can work around this. So an application like this that already has all of our authorizations could effectively serve as a dead man's switch. That is like if you fall off the pedal,
23:00
the train stops. So maybe I could list beneficiaries like Todd and Brandon, and after, you know, I go absent for a certain amount of time, maybe it sends me little tickler emails every month, and if I go away for 60 days, it knows to add Todd and Brandon as owners on all of the projects that I've authorized it to. By the way,
23:21
I like to come up with a name for a thing before I build the thing. I'm not going to build this, but I like to call it somebody-please-make-this.io. Really, somebody please make this. I would be very appreciative. So that's all that's just about dependencies that are like, you know,
23:41
binary dependencies or runtime dependencies that we package with our applications. What about all these cloud services that we use, right? This slide is a slide of services that are going to shut down someday. Next slide. Can any centralized service, probably written volunteer on less money,
24:02
you know, if the corporate ones are all going to shut down, what about the purportedly open ones? And can they be truly open? Like a maximally open system is rife for abuse, but if you make it too curated, then it could exclude people and limit the amount of contribution. And I ask this, of course, because almost all open source infrastructure is entirely centralized,
24:21
out of convenience, right? Because the march of progress just wants to make it easier and easier to adopt stuff. It's a hard question to answer. How do we decentralize this? You know, a lot of people in the room contribute to RubyGems, and I think they do great work, and I think they're underappreciated. But what if RubyGems were to disappear? How many businesses would that affect? Do the businesses relying on RubyGems realize that as liability?
24:41
What if NPM were to fail and lose a month of backups? How many things would no longer be able to install and work? I ask the question because I want to know, like, what would a decentralized dependency service maybe look like? We have BitTorrent, apparently, right? We have cryptography. It seems like the sort of thing that we'd be able to figure out without having a single point of failure.
25:02
Speaking of single points of failure, one of my favorite things to do is wait until the next time GitHub goes down, because people freak out on Twitter, and then they make the kind of snarky joke of like, well, you know, it's distributed, so why do you guys care? Because I can still work locally, and in fact, I can SSH to my buddy, and we can still collaborate.
25:20
Good thing that's all we use GitHub for, right? It's not like we use GitHub to pull down our dependencies, or we use GitHub to test all of our code, or we use services that depend on GitHub to analyze all of our code. We even have most of the continuous delivery services that ship our code also depend on GitHub being up, and even if you finish your work,
25:41
your next issue is in GitHub issues, because that runs on GitHub, and you can't find the next thing to do, because now it has also co-opted all of our project management. So how can we connect numerous services to our code, which is admittedly the source of record, the thing that everyone's going to want to integrate with, but still avoiding that single point of failure? Well, fortunately, Git is very fast and very portable,
26:02
and it seems to me like a distributed transport layer or mechanism that would let these people hook into it without having a single company needing to be up for everything that we do to work seems possible. Somebody please build this. Let's talk about trust. So open source requires adoption, obviously.
26:21
It's optimized for it, and in order to adopt something, people have to trust that it's good. Now, there's explicit trust, which is the dependencies that we directly depend on, but there's also implicit trust, the stuff that they depend on. We just sort of trust that the people we trust are trusting the right people, et cetera. It's a big web. But as a maintainer, how do I get people to trust me
26:41
and use my thing? The answer, of course, is marketing. Let's look at marketing over time in open source. So consider Linus Torvald's 1991 announcement of Linux. He did it on the Minix mailing list. Here it is. And look at it. Got a few paragraphs. But no catchy name.
27:00
He makes a self-deprecating remark in the first line, and then he goes totally off-message and talks about how to only work on his niche particular hard drive. This is bad marketing. Linux, if it were announced today, this thing runs most of the servers in the world. It wouldn't even make the front page of Hacker News if it came out today. I think that's reason for pause.
27:21
A decade later, the ANT project for Java, this is trying to get into a lot of big corporate environments that don't trust open source. Look at all the things they had to do. They had a fancy logo. Look at all this website stuff. They had a mission statement. And then, of course, Apache, they had a foundation affiliation to try to confer trust.
27:41
But the more dependencies that were hovering up over time, the less time we have to vet them. And so nowadays, the standard GitHub markdown readme is expected, and here, you know, you expect a quick, catchy intro, a whole bunch of real easy steps to get started, and then some mostly green badges up at the top to tell you things are working. And it's gotten even further because now there's all these corporate-backed
28:04
sponsored projects. This one is Yeoman, where a diverse group of engineers are building a rocket with gradients and an authoritative tagline, and they even have, like, a one-liner install with a wizard that walks you through how to generate a thousand different types of projects. It's a bad idea. Separate talk.
28:24
Everything in open source is optimized for adoption. It's the natural selective pressure that we're under as maintainers. You know, so we have all these, like, rock stars. People in the community we look up to, like when Aaron Patterson releases a new gem, I trust Aaron. I know he's, you know, good at programming codes,
28:41
and so I trust that his gems are going to work. I could look at how many stars or how many forks a project has. I could look at how much activity there is and how many open issues. I can see how many times that project's been downloaded. Now, even though 90% of those are Travis CI downloading them over and over again, my lizard brain tells me it's still important. Semantic versioning. We went, like, 40 years without any sort of, like, you know,
29:01
proper formal way of versioning software, but why did we need one now? The answer is we have so little time to, like, vet our dependencies that we need to know at a glance, like, is this ready for use or is this safe to update? And who's got time to vet transitive dependencies? Nobody. The more people that you explicitly trust, the more people you don't even realize that you're trusting.
29:24
I encourage everyone to recognize, too, that all these projects are marketing to you. Like, I write open source and I want people to use it because I love it and I'm passionate about it. I want you to engage with my brand and use my stuff, but I hope that you have a head of discernment on and you realize that every new thing you pull in adds complexity to what you're doing and more stuff can fail.
29:44
Speaking of failures, let's talk about security. I actually have been, for a long time, a believer that you can do worse than security through obscurity. You could have a whole bunch of stuff up in the open and nobody working to secure it. Of course, the Free Software Foundation would point out, well, open source code is accessible to everyone, right?
30:00
The cathedral and the bizarre argument of, like, many eyes and shallow bugs. But that's only technically true. People have to read the code for that to be true. So who reads the source code? First of all, there's the people who claim to read the source code and then there's a tiny minority of them who actually do. There's people who fork projects, but so few people who fork do anything. There's the people with commit rights,
30:21
but then the actual committers are usually only a handful of that. There's people who send a pull request, but only a handful of pull requests are anything more than just drive-by single tactical things. And then there's the people who are hunting for exploits. And this is the only community where 100% of them you can rely on reading the code. So in absolute terms, an uncomfortable proportion of people reading the source code
30:42
are actually out to exploit it. Bash is a computer program. It's in everything. In the Internet of Things, all those things run Bash, like your smart fridge, et cetera. So it's kind of unfortunate that Bash had a terribly big security exploit because a lot of those things are not upgradeable and patchable. Somebody went and looked at that code,
31:01
and what they saw shocked them. It was global variables everywhere. There was all these void methods that took no arguments. Basically, that means that all that method could possibly do is A, nothing, or B, muck with global state. And then here's the vulnerable function. I'll give you a second. If this looks like nonsense to you, I agree.
31:23
We're going to read a line of open source as a group. This is the beginning of a for loop. And I look at this, and I'm a Ruby, so I don't spend a lot of time with for loops. But I look at this, and I'm like, why is that called string index? Why are they doing assignment in the second thing instead of a Boolean operation? Why are they incrementing in the second clause instead of the third clause?
31:44
Surely they did that for a reason, I think, because I would write that like this. And so I always trust, I always assume that the person who wrote some other code is as smart or smarter than me. So clearly they must have done that for a reason until I can prove that they didn't. And so the amount of cognitive depletion that occurs as I read this method was so great, I only got like halfway through it.
32:03
So the Free Software Foundation, who maintains Bash, says the solution is not proprietary software. The solution is to put energy and resources into auditing and improving free programs. And I think that's a pretty agreeable thing. So who here wants to audit the quality of code that literally everybody depends on?
32:21
None of you, right? Why is that? It's because as a library becomes more popular, the importance that we audit that it works and that it's safe and that it's secure goes way up. But our motivation at an individual level to audit it goes way down. If I use a gem and only three people are using it, I'm probably not gonna trust that it works, so I'm gonna read the source code.
32:40
But if I'm using Bash, I'm not gonna think I should probably read this. I'm gonna assume somebody else did. The downside is that everyone else is assuming the same thing and no one's actually reading it. This is an example of tragedy of the commons. And in this case, this is an example that it's nobody's problem until it becomes everybody's problem. And at that point, it's too late. So my hope is there's this recent group together.
33:01
I think that companies can solve this by investing because they have the money. And if they collaborate, you know, the to-do group is an example that they came together. They want to form some norms about open source governance. One of the things I hope they look into is sharing notes when they're auditing the security of stuff, working together to patch exploits. I don't know if that's one of their goals, but I think that's probably where the solution lies.
33:23
If you work at a smaller company, it's your responsibility, I think. It used to be the case that we were trying to get open source into the door at wherever we worked, and so we didn't want to say anything negative about it. But now everyone needs open source, and I think it's our responsibility to tell our bosses that open source is not a free lunch. It's not some altruistic thing to make sure that it's working, right?
33:43
It's a massive source of potential liability, and it's worth vetting it out. I want to share some thoughts about our interactions as people, as well as where I think the future's heading. So unfortunately, all of these stick figures that I drew painstakingly were a lie,
34:01
because what they intimate is that open source is a thing that happens between people face-to-face in human interaction. But it's not like that. How we actually communicate is email, GitHub issues, Twitter. Almost all of it is just asynchronous text. You know, we're all no more than an avatar, a username, some emoji now and then, and text on a screen.
34:22
That means that when you get riled up about open source, no one can hear you scream. I'm screaming all the time, and none of you ever come to say hi. And that's a big part of the problem. Evie had a tweet recently I liked a lot. She said, it's effed up that a lot of modern discourse is optimized for whoever has the fewest feelings
34:40
and the most free time. Who would design that system? Us, apparently. So when there's uncertainty, ambiguity, or disagreement in a team, and if that leads to simmering disdain, is email the right use, the right communication channel? Probably not. I think that it makes more sense
35:01
to escalate to a higher fidelity form of communication. So if there's ambiguity, maybe real-time text makes sense, could clear things up. If you're having a voice conversation, you can emote and empathize with the other person. If you're making somebody cry, you can see it on a video chat. And if all else fails, you can always meet up in person. And there's something about human biology
35:21
that when two people get in a room, they tend to walk out with some kind of compromise, and that doesn't seem to be true on Twitter. I don't know why. Also, this strategy can be great troll repellent, right? Trolls get their power through anonymity and the perceived lack of consequences for their action. But if you level up the fidelity of the communication and the discourse, that means that their anonymity is also going to erode,
35:42
and they lose that. And so if somebody's trolling you, you're like, all right, cool, let's have a hangout. Suddenly, they go away. So what I'd love to do is, like, make opting into these higher forms of communication easier. So if I'm on a GitHub thread, I'd love a button that would open a new chat in Gitter, which is a tool for chatting on GitHub, or build a live example in place,
36:01
like through JSBin, or schedule a pairing session with Screenhero or AirPair, or hop onto my Nitrous I.O. box right in place. I think anything to encourage us to have those more real communications, the better. I also want to think about where the future's heading, because I showed a graph that it's going to get worse.
36:21
So on that graph, I said, no, okay, we're here, and, you know, it's going to get worse before it gets better. So what does this look like? And I was stewing on this while Gogaruko was happening, and I was thinking, like, if we extrapolate from our current culture of dependence, where would that take us? And I didn't come up with a lot of great answers, but then I saw this tweet.
36:40
Yehuda was giving a talk about the Rust programming language, and he says, Rust enables a whole new generation of high-level programmers, us, to write systems-level code. And my initial reaction was, ugh. We're this huge pyramid of stuff we don't understand. That's scary. But as I explored it and started working on this, I actually realized I'm okay with this.
37:02
There actually might be some benefits to this, and let's talk about those. So if you think of innovation over time, low-level system innovations happened before high-level ones, naturally. You need low-level systems before you can have high-level ones. And then they crept up. In the last 20 years, most of this open-source innovation that we've experienced has happened in high-level programming languages. Now, only very recently
37:21
have we started reinvesting in low-level systems like Go and Rust. But today's open-source dependency culture, really, is steeped in a lot of assumptions about high-level code. And so the question is, if we were just to cargo-call that down to low-level systems engineering, how would that translate? How would it work? When I think of my systems programming friends,
37:42
they tend to be very conservative and cautious. And that's because they're a mix of isolated from innovation and just curmudgeons. Put a slightly nicer way, that means that there's a certain amount of accidental or incidental cautiousness, a certain amount of intentional learned cautiousness, and maybe I can learn from that cautiousness, too.
38:01
Embedded in real-time failures have very grave consequences in some cases, and that's probably where that cautiousness comes from. So if you think of a high-level system, what's the worst that could possibly happen? Healthcare.gov starts failing and people can't get their insurance. That's really bad. Yeah, that's like 60 night lines in a row
38:22
of drama for our nation. So let's take the same technology and put it in charge of this thing and see how that works, right? Low-level systems, there's way more rigor required in terms of fault tolerance because people can literally die if you don't get it right. My systems friends have a different perspective
38:42
on dependencies than we do. They view adopting a dependency as outsourcing our understanding of how to do something. That means that if we have an application responsibility, that orange blob over there, and some particular dependency, we can view that dependency as solving part of our application's problems for us. We still know what it's doing and why,
39:02
but we might not know how, and that represents what I call understanding debt. Understanding debt is typically paid down by iterating. Any dependency we have, we learn about it through usage and through changing, you know how it works and what we need to work around, what's not great about it. But if iterative releases aren't possible,
39:21
then you shouldn't outsource that understanding. Like if we think of high-level systems versus low-level systems, a high-level system is highly centralized. It might only run one instance. Low-level systems tend to be built, fabricated, shipped, and then they're out the door. High-level systems have very short lifespans, typically. Low-level systems might have to live and be raided for very long periods of time.
39:41
High-level systems usually have a full staff maintaining the thing for its entire life, but low-level systems might just have a support chain. High-level systems usually have way more than enough overhead in terms of memory and CPU in order to absorb the impact of some inefficient dependencies, but low-level systems have very hard and real caps. High-level systems are highly connected.
40:00
We have all these tools to automatically deploy patches and new stuff, but low-level systems might have intermittent connectivity or no ability to be patched at all. So clearly, we've made it very easy to iterate high-level systems, but at this point, it's still very hard to iterate low-level ones. So these concerns require that systems engineers have a deeper, upfront understanding
40:22
of what they're building. If you consider, like, if you were to graph out the depth of understanding necessary to be successful in a high-level web app versus, say, like a low-level plane control, like a landing gear or something, you only really need to know how browsers work, HTTP, a little bit of JavaScript, the request-response lifecycle. But you might need to know a lot upfront
40:42
about how planes work and everything that can go wrong and everything that you're gonna be plugged into, and you need to account for that on day zero. So this traditional, like, you know, we write a handful of little tiny domain objects on top of this mountain of open-source stuff that we don't understand, that might be an appropriate level of depth for a high-level app. But if we were to simply cargo-cult that
41:00
over to a low-level system, there's this danger zone in the middle, right, of stuff that we don't understand and is inappropriate and might put us at risk. And so systems engineers have a couple options They can either build more, richer, deeper domain stuff, or they can go through a process of qualifying those tools and literally rigorously proving that they work.
41:20
This reflection caused me to realize that modern tooling is just a product of high-level web development. You know, if we look at this chart again, this represents just today's perspective on how dependencies work. And if we were to do the same thing at the low level, rather than worry about cargo-culting ruining systems code, I'm pretty sure they're gonna still make sure that stuff works, maybe together, collectively,
41:42
we can have a new and broader perspective on what is currently our, you know, culture of dependence. In fact, I'm hopeful now that systems innovations, you know, bringing the same convenience that we have in open source to low-level systems, they might reciprocate some cautiousness,
42:00
desperately needed cautiousness, as well as emphasize the importance of understanding the things that we're shipping. So we covered a lot of ground in these four areas today, but I want to part with the message that I think that the open source world can be a lot better and there's a lot of room to improve and it's all actionable and most of it's fixable. You know, companies, I think that we need to do our job
42:21
of convincing companies that they have a responsibility to be contributing back and to be auditing the security of stuff. I think startups, you know, need to be cautioned that there's no such thing as building a real app in a weekend and being done. Maybe that means that they should invest the time in understanding the things that they're standing on top of
42:40
or if they have to get out the door in 30 or 60 days or whatever, understanding that you probably want to throw away version one when you go back to rebuild version two after you've validated after you've been successful. I like to call this the slow code movement. Individual maintainers, I think that they can learn a human lesson from this that is
43:01
aggressively try to pull in new owners and contributors as early as possible while there's still excitement to do that. And then I think we can solve with tools, make it easier for people who want to participate and get involved in open source, make it very explicit and invite them as opposed to just waiting for them to work up the courage to email somebody.
43:20
So once again, my name is Justin. You know, please say hello. I come from Test Double. We're an agency of really terrific software developers and so if your company needs developers and some help, please let me know and I'd love for our great people to work with your great people. Like everyone else, we're hiring too, so feel free just to drop a line and join at Test Double.
43:43
I'd love to chat with you during a break today and if you're too shy for that and you just want a free sticker, I got stickers too. And most importantly, I just want to thank you all for your time and thank the organizers for having me. I'm really, really humbled and privileged to be here. Thanks.