DjangoCon US 2017: Lightning Talks Day 2
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
Title |
| |
Title of Series | ||
Part Number | 12 | |
Number of Parts | 48 | |
Author | ||
Contributors | ||
License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
Identifiers | 10.5446/33197 (DOI) | |
Publisher | ||
Release Date | ||
Language |
Content Metadata
Subject Area | |
Genre |
DjangoCon US 201712 / 48
2
5
6
14
16
23
26
30
31
32
34
39
43
48
00:00
Web pageError messageGenderWeb pageCodeVideo game consoleDemo (music)Link (knot theory)Software bugCASE <Informatik>Electronic visual displayRadical (chemistry)Computer programmingComputer animation
01:03
Generic programmingUniform resource locatorWebsiteMassView (database)Template (C++)Endliche ModelltheorieException handlingDirectory serviceInformationWebsiteType theorySoftware bugIntegrated development environmentHome pageLine (geometry)Open setComputer animation
02:26
Uniform resource locatorAuthorizationInformation securityError messageMaxima and minimaRankingServer (computing)Link (knot theory)Software developerWeb pageError messageOpen sourceCASE <Informatik>Sheaf (mathematics)Projective planeInformationException handlingGodComputer animation
03:11
Web pageError messageElectronic visual displayProjective planeOpen sourceForestProcess (computing)Error messageForm (programming)Right angleWeb pageImage resolutionProduct (business)Link (knot theory)Revision controlTheorySoftware bugComputer animation
05:01
Web pageElectronic visual displayError messageBuildingTwitterGoogolPoint (geometry)Uniform resource locatorData managementServer (computing)Computer fileInstallation artProgrammable read-only memoryInterpreter (computing)Scripting languageOvalString (computer science)2 (number)Duality (mathematics)TwitterMereologyPoint (geometry)Physical systemBitWordTemplate (C++)Software developerServer (computing)Product (business)Set (mathematics)DatabaseFluid staticsMatching (graph theory)BootingUniverse (mathematics)AreaSequelVirtual realityComputer fileKey (cryptography)Cartesian productProjective planeWeißes RauschenData managementEmailComputer programmingRight angleDemo (music)Process (computing)SoftwareMultiplication signAliasingGreatest elementUniform resource locatorSystem callRegulärer Ausdruck <Textverarbeitung>Flash memoryMobile appInstallation artPersonal digital assistantVirtualizationComputer animationSource code
10:30
Directory serviceInstallation artCache (computing)PixelVideo trackingDataflowWeb pageStructural loadFunction (mathematics)Computer-generated imageryEmailOpen setLine (geometry)Software testingControl flowDatabaseMetric systemTrailExtension (kinesiology)Computer networkNumbering schemeWebsiteHuman migrationEndliche ModelltheorieSocial classData modelField (computer science)Operations researchTable (information)Price indexMetric systemUniform resource locatorCoefficient of determinationPixelTrailInternet service providerMedical imagingProxy serverSoftware testingEncryptionDatabaseClient (computing)Revision controlDifferent (Kate Ryan album)Structural loadEmailDataflowCodeBit rateSoftwarePhysical lawSequelUniqueness quantificationElectric generatorMoment (mathematics)CodeGraph coloringAreaLibrary (computing)Extension (kinesiology)Product (business)PlastikkarteOpen sourceParameter (computer programming)Web pagePhase transitionInformationCodierung <Programmierung>Human migrationLine (geometry)Heegaard splittingTheoryPhysical systemWebsiteProjective planeTask (computing)Order (biology)Endliche ModelltheorieField (computer science)Key (cryptography)Archaeological field surveyLink (knot theory)Numbering schemeUnit testingFunctional (mathematics)MathematicsOperator (mathematics)Cartesian productMixed realityMobile appOpen setFront and back endsComputer fileQuery languageMultiplicationComputer animation
15:49
Table (information)Price indexDean numberHuman migrationSubject indexingWebsiteTable (information)Multiplication signBit rateBlogCoefficient of determinationQuicksortPhysical systemReverse engineeringError messageHuman migrationVariable (mathematics)Axiom of choiceDensity matrixStructural loadAlgorithmCASE <Informatik>SequelOperator (mathematics)Block (periodic table)AreaRight angleDatabaseResultantSlide ruleCellular automatonXMLComputer animation
18:02
Cohen's kappaLaceCodeTablet computerRevision controlEndliche ModelltheorieReal numberImage registrationDefault (computer science)Template (C++)UnicodeSoftware frameworkData modelVideo gameVector potentialWell-formed formulaArithmetic progressionFamilyBitMultiplication signCartesian coordinate systemInternet service providerSet (mathematics)DatabaseDefault (computer science)Key (cryptography)Real numberCodeVideo gameView (database)Image registrationSoftware bugPhysical systemEndliche Modelltheorie1 (number)QuicksortFamilySlide rulePlanningOperator (mathematics)Form (programming)Human migrationTable (information)Single-precision floating-point formatField (computer science)Different (Kate Ryan album)Repository (publishing)Inheritance (object-oriented programming)UnicodeParameter (computer programming)Limit (category theory)Functional (mathematics)Fitness functionTemplate (C++)Graph (mathematics)Direction (geometry)MereologyExtension (kinesiology)Profil (magazine)Mobile appCodeCoefficient of determinationVolumenvisualisierungFood energyRenewal theoryOnline helpSocial classParticle systemGenderWebsiteCASE <Informatik>NumberCore dumpCommitment schemeGroup actionWordSign (mathematics)SoftwarePlug-in (computing)Execution unitInsertion lossMedianRevision controlComputer animation
25:33
Direction (geometry)Process (computing)MereologyCore dumpMetric systemCondition numberUniverse (mathematics)Self-organizationFlow separationPosition operatorArithmetic progressionMultiplication signProduct (business)Message passingOffice suiteFamilyStaff (military)Decision theoryBranch (computer science)Point (geometry)Archaeological field surveyDigitale VideotechnikDegree (graph theory)Roundness (object)Different (Kate Ryan album)Projective planeComputer animation
28:45
Covering spaceRepresentational state transferVideo gameEntire functionWindowWorld Wide Web ConsortiumPoint cloudData storage deviceRaw image formatTotal S.A.Hyperbolic functionThermal expansionSimulationRemote procedure callSource codeData storage deviceChannel capacityBefehlsprozessorWordEntire functionScalabilityThermal expansionRaw image formatCuboidRepresentational state transferTotal S.A.DatabaseDecision theoryVideo gameConnected spaceDifferent (Kate Ryan album)Set (mathematics)PermutationMathematicsSpecial unitary groupService (economics)Point cloudProduct (business)Web 2.0RandomizationWeightComputer animation
30:22
Cloud computingWebsiteOracleServer (computing)EmailWordDisintegrationSource codeRepresentational state transferOpen sourceSoftware frameworkBuildingEndliche ModelltheorieTotal S.A.CodeDemo (music)TwitterFilter <Stochastik>Serial portRepresentational state transferWebsiteCloud computingServer (computing)INTEGRALGenderPoint (geometry)WordTable (information)SequelClient (computing)Faculty (division)DatabaseSubject indexingResultantComputer programmingBookmark (World Wide Web)SatelliteVideo gameMereologyEndliche ModelltheorieSoftware frameworkOracleFile formatDecision theoryDisk read-and-write headException handlingCellular automatonEntire functionSource codeComplete metric spaceLOTUS 1-2-3Set (mathematics)Canadian Mathematical SocietyAdditionCodeMixed realityWeb 2.0Division (mathematics)Multiplication signComputer animation
33:44
Menu (computing)CodeExecution unitVariable (mathematics)RootDemo (music)CodeWordRight angleVolume (thermodynamics)Computer animation
34:26
CodeImage resolutionCASE <Informatik>Process (computing)Digital photographySystem callDisk read-and-write headShared memoryShape (magazine)Cone penetration testSocial classHexagonRight angleVapor barrierMultiplication signClassical physicsCodeComputer animation
36:17
WhiteboardFrame problemCuboidFrame problemProteinDigital photographyReduction of orderControl flowWhiteboardSuite (music)Source codeMultiplication signPoint (geometry)Group actionOpen setBranch (computer science)OvalFilm editingRight angleDrop (liquid)Different (Kate Ryan album)Level (video gaming)Perturbation theoryStructural loadSpherical capCone penetration testInstance (computer science)Message passingSlide ruleInferenceElectronic mailing listComputer animation
40:42
XML
Transcript: English(auto-generated)
00:01
All right, so I'm Felipe, this is Flavio. We are both from Brazil and we work at Vinta.
00:21
We are partners there. And we are going to talk to you about FixMyJungle. So this started when we used to program in Angular 1 and Angular 1 had this thing where whenever it displayed an error, you could click on the link in the console and it took you for like a webpage
00:43
where you could see something about error, how to fix it, what was it. And we thought that was pretty good. So we decided to do it for Django. And that's when we created FixMyJungle. And we are going to show you in a live demo now. Okay, here is some code that has a bug.
01:06
Okay, so here is like, hope you can see it. Hey, I have a view with a template that does not exist. And then on the URLs, I just try to use that view. And when I go to local host and open that view,
01:25
it shows that the broken template, the plate does not exist. And it says that FixMyJungle may have a solution for this exception. When we click it, it goes to FixMyJungle website, filtering by the exception type we just saw by which model it was erased,
01:43
by which line. And then we can click and we are able to see more information about the exception, the traceback that generates that exception. This traceback is like sanitized because it doesn't show the environment before the site packages.
02:02
So it will not show your home directory or something like that. And then show how to help reproduce this bug and how to fix it. You should verify if the name is spell errors, if the template name is correct, if the folders are correctly configured on templates, setting, everything.
02:23
To install FixMyJungle, you just need to keep install FixMyJungle and go into the settings and add here to installed apps. And by doing this, we override the full error page for Django development server to be able to have this link that I showed to you.
02:43
If the section doesn't exist in FixMyJungle, you can contribute with it. So for example, here another case where you can add the error to FixMyJungle. And when you click it, you see that we already got the exception information that you just saw and we don't have it.
03:01
And you can add some information about how to reproduce and someone else can help you on how to solve this. All right, so FixMyJungle is a open source project. It's on GitHub. You have to go to slash FixMyJungle all together.
03:23
And the history of the project's pretty much, we created this two years ago. We use it internally at Vinta. But the thing was, we wanted to gather some initial data. So like after that, we could publicize it more.
03:41
That's actually why we couldn't get this out because we didn't got much engagement even inside the company. And we think that was pretty much because of the way we are doing stuff. So nowadays we show the link. It would be good to actually show the full resolution
04:03
in the Django error page. So next steps will be, we got a couple bugs. We kind of stopped working on this a few time ago. So we do have bugs. We need to add support for,
04:20
maybe there's some bugs on Python 3 and Django, the latest versions. So yeah, we need to display how to fix the errors inside the error page. We would like to have, so you could submit the, have that form for submitting errors
04:40
inside the Django error page as well. And it would be good to have ticketing so anyone could contribute. So we will be here for sprints and we would like to work on this. And so if you want to help us out, just talk to us. Thank you. Do we have time?
05:03
All right, so we got a few seconds. What is it? Oh, shit. You've probably started. Sorry. Yeah, so as I said, we are from Brazil and we have a Python there.
05:20
It's called Python Brazil and it's going to happen from the 6th to the 11th of October. So if you want to come, please let me know. I can help you out with anything. Unfortunately, the talks are all in Portuguese so you might have some trouble. But you are welcome. All right, thank you.
05:43
All right, yeah, so the Zen of Django is a little package I put together. So just a little bit about me first. I'm an instructor at a boot camp in Houston called Digital Crafts. I'm also using Django to build a Google Assistant voice app. So if anyone wants to talk about that, come see me after the talk.
06:02
If you said those little words, probably 80% of you on your phone actually would bring it up and follow me on Twitter. I'm the pizza panther. So what is the Zen of Django? It's just some tools I put together to streamline and get you started with Django a little bit easier. So I have a little bit of flexibility
06:22
with the curriculum I teach. So I'm a Django guy. I wanted to teach my boot campers some Django. So I probably, part of it was my fault. I went through it too fast. But I started seeing some of the pain points of getting newcomers to start Django. Some of the pain points I found,
06:42
you need regex and URLs. That's not necessarily an easy thing for a newcomer coming to programming. And some of this stuff too is just with developing Python. It's not necessarily a Django thing. So you gotta worry about package management. You gotta worry about not committing your secret key.
07:00
Even I've done this after 10 years of development. Thankfully, there's that guy who follows all the public projects and makes sure you don't commit that secret key. I got that email, so thank you, whoever that is. There's no built-in production server, static files.
07:20
You gotta worry about how you're gonna deploy that. And then switching from your production to your development settings. So I created this little package called Zen of Django to kind of put this all together, make it easier to start out so that you don't have to necessarily worry about some of these things. So the first thing, simplified URLs.
07:40
So instead of using regex, I stole the flash style. And basically, you can mix and match between the Zen URL and the Django URL. It's actually just converting it to regex for you. But I think that's a lot easier to understand. I grew up on Perl, and I eat regex for breakfast.
08:00
But I still know that I don't really wanna use regex a lot. So I think this simplifies the URLs a lot. Let's see, next thing, package management. So you get a requirements.txt, automatically generate it with whatever packages are already there. I've also included pip save. So just like with npm, where you can install a package,
08:23
and then it saves it to your package file or requirements.txt, so that's built in. Auto-generate it, gitignore. I had people, again, committing their database, their SQLite databases, so just get rid of that problem.
08:43
.env with your secret key, again, is initialized for you, so you don't have to worry about committing your secrets. And deployment. So I think micro-whisky is pretty much the preferred method, so I built that in from the Django micro-whisky package.
09:02
White noise for static files, that's all built in, and then the project template is automatically set up to switch between your development settings and your production settings. And you can see down here at the bottom, I put in a dev server, so you kinda know
09:20
what you're running. That's just actually an alias to run server. And then when you wanna switch to production, you just run the prod server, which will run Django micro-whisky for you. All right, so let's see if I can have a demo fail here. So I'm gonna go through it here. So we'll do a Python virtual env here.
09:53
Do I get time for network calls here that are slow? Nope. All right, so I made my virtual environment.
10:03
Now I'm gonna do pip install djzen. And get that installed. You can see it's installing a bunch of stuff for me. And let's see if I can get to the final thing. So you can just start your project
10:22
instead of using the Django start project. And I kinda issue some recommended settings, and I guess I showed them to use the test. So we'll just use that. And there you go, you got your get ignore, you got your requirements, and then if you can see it
10:43
here, ding-dong, you got your envs and your secret key. All right. My name's Richard Latimer, creator of Ptrack, along with Sara Ellis from Indeed. So this is a tracking pixel library.
11:01
What is a tracking pixel? It's a image tag, typically one by one pixel, transparent, and there's a unique URL that's associated with that. Here, this is kind of the two typical flows, you have this small image that loads on an email or a webpage, when it's loaded, when the user visits or opens the email, you then execute a logging function
11:23
and you log that this user visited the page. So this is really useful because we like to track email open rates, you can log that data, and then you can see click-through rates. So then, if you have your own tracking pixels, you could do AB testing, so I could try one subject line versus another subject line, split it among my users,
11:41
see who's opening more of those emails. With the goals of Ptrack, looked at various different ways people are doing tracking pixels, and tried to do it the Django way with the best of those ideas. So one, there's no database, the data's based, we put it inside of the URL. You have flexibility with the metrics, so you're not based on a unique ID Ptrack gives you,
12:02
you get to decide the metrics. It's secure, so these metrics are encoded and then encrypted into a URL, and then it's fast when this decryption and decoding phase, it happens in our production settings within 80 milliseconds or lower. So here's what a typical flow would look like for Ptrack.
12:21
So you load the Ptrack module, you load the Ptrack template tag, you can have arguments, so here we have our test name, email test one, and then maybe you wanna find some distinct information about users, so you could pass in the user ID, you could do some distinct email open rates, et cetera. Then what links that template to your logging
12:41
is you instantiate a version of a tracking pixel, there's a callback routine called record, you have access to your request, so you could see through that info what mail clients are accessing you or what users are visiting your webpage. You can access your arguments and you can access your key value arguments. And finally, these, say here, custom tracking pixel,
13:03
you register with Ptrack, you can actually chain these together, so if you wanna split up your code, how you do your tracking pixels, it's up to you. There's no guarantee on the order they execute, but you can chain them together. For a few extensions, say you wanna do internal email tests,
13:20
so you wanna say, you know, people aren't opening my surveys, I wanna do email subject tests, see what people are doing. You actually can do this, so most mail providers proxy your images, so you can't actually just host it from your internal network. What you can do here is you can deploy a mirror version of your site, but you know, just a thin client,
13:42
so it's merely there for tracking pixels, and you can define your app URL where the tracking pixel would be loaded, and then you share a secret key for encryption and decryption of the metrics. If you don't wanna use the encryption scheme that comes with Ptrack, we're using PyNACL, you can define your own encoder
14:01
and use it there instead. And Python 3 support's coming soon. This is the first Django project indeed has open source, so was pushing to get that up before coming here, so after that, we're working on Python 3 and hoping to contribute more to the community. So here's our GitHub. You can actually pip install using Django-Ptrack
14:22
and wanna open if there's any questions. All right, thank you. Thank you. All right, hi, my name is Johan Heller. I work for rover.com, and I'm here to talk about non-locking migrations and Django. So everybody likes dogs, and I like Django,
14:42
so let's mix the two things together. So we got a dog model, and it just has a name field. We're gonna add a birthday field to that, because dogs love receiving birthday cards in the mail. It's a well-known fact. And it has a database index, so we can make query on it fast. Making a migration in Django is super easy.
15:01
You just call make migrations. You can pass arguments through if you want more custom, but that works. It will spit out a migration, looks like this. It's actually a really, really handy feature of Django, the migrations subsystem. It keeps track of dependencies. It has all the operations it's going to do. It's database independent, so this can be run across
15:21
multiple different database backends. Really handy feature. So then you actually wanna run your migration on production, so you run migrate, and Django will take that migration file and make this SQL. This is database dependent, so it will change the SQL slightly. The only difference between Postgres and MySQL here is the coding, but it's pretty standard SQL,
15:42
so we're gonna go ahead and run that. We'll give that to MySQL. That's the database system that we use at Rover. And MySQL does this with that. So what happens there is both the alter, adding the column and adding the index, lock the table for writes while it's doing that,
16:03
and if you have a very large table, the table will remain locked for writes for a long time, which means no users can add dogs during that time, which makes for really unhappy users, because people love to talk about their dogs and add them to a website. So what can we do about that? So MySQL 5.6 added some features
16:21
for supporting sort of non-locking operations, and how we take advantage of those is we can just alter the SQL that Django generated for us to add a couple more parameters so that it doesn't lock. Just a quick side note here. Alter table uses commas, and create index doesn't.
16:42
And if you don't get that right, it errors with syntax errors, and it just infuriates me that that's the case. So going back to the actual full SQL here, the things we added are the algorithm in place, which tells MySQL that you wanna use the in place sort of upgrade of the table,
17:01
and not a copy operation, which is the sort of other choice. A quick note, it sounds like in place would be faster than copy. It's actually not, it's actually slower, and adds more load on your database, but it has the advantage of it doesn't lock things. So you can expect worse performance for the migration, so take longer, and they'll be doing more work, but they won't lock. Lock none here is sort of just saying the same thing,
17:22
like please don't take any locks. It doesn't actually mean no locks, it just means less locks for less time, because it's MySQL, so it sort of doesn't do what it says. And a note here is if you run it like this, and it has to take a lock to do the thing you want it to do, it will just fail. It won't silently downgrade to taking locks,
17:42
which is actually really handy. All right, so how do we use that with Django? So Django has a run SQL migrations operation, so we can just specify our own, the SQL and reverse SQL are variables that are off the slide, but they just are the same things we saw before in the reverse of that, and you specify the state operations so the migration system knows what's going on,
18:03
and then you're like, oh yeah, awesome, this is working really well. And then you realize that this is part of the docs describing all the limitations of this system. So basically every single operation has a different set of limits and restrictions on how it works.
18:20
I can't fit it on a single slide even though I think it's really, really tiny. The sort of big ones to point out are it requires InnoDB, it requires MySQL 5.6 or later, it requires that your table have been created in MySQL 5.6 or later, or you have done a copy operation on that table, which you also can't, MySQL won't tell you if it's done that, you just have to know based on dates and things,
18:41
it's pretty annoying. Yeah, this doesn't exist yet, well it exists, like you can go there, but there's nothing there. But this is what I'm planning on sprinting on, which is adding the migrations system to, or adding a database back in that adds these parameters for you.
19:02
At the end, just a shameful look, I work for rover.com, and so if you like Python, Django, or dogs, come talk to me. So hi. Some of you may be worried about starting your first Django application
19:21
or writing your first bit of code. I'm here to tell you that I have at least one that I've been maintaining for over 10 years, and I'm still worried, but it gets better and you learn a lot of things. So let me tell you a little story. I wanna take you back to a simpler time when the world made sense, it was the year 2007.
19:41
And Django had been out for about a year, and people kept coming up with these, so people kept coming up with these cool things that they figured out how to do with Django, and they wanted a way to share it, but Python packaging wasn't that great yet. So I built this website called jangosnippets.org, which still exists but is maintained by other people, and was a place that you could share
20:00
these little bits of code that you'd figured out. But I needed a way to let people sign up for it and have an account there. So I did something silly, I started writing a Django application. This is February of 2007, the initial commit of an application now known as Django registration. This has been around for a while,
20:21
it was originally Google code and subversion, then it was Mercurial on Bitbucket, now it's Git on GitHub. Django registration was built on a thing that I was really pushing hard at the time, which is the idea that a Django application should do one thing and do it really well. So in this case, it was just about signing up user accounts, nothing else, not filling in a profile,
20:43
not letting them connect other things, just sign up a user account. Unfortunately, there are some things that are hard. So back then, Django did not have class-based views. If you wanted flexible, extensible apps that people could add on functionality to, you had to add a bunch of arguments
21:01
onto your function-based views. I tried to do default templates, eventually I tried to do custom user models, and also I had a dose of real life in the middle of it. So this is the sort of thing you used to have to do with a function-based Django view. You just kept piling in more and more and more keyword arguments there. Some of the worst that I've seen
21:20
is if you go back and look in the Django repository, just before we switched over to class-based generic views, some of them have definitions that look like this because of how many different keyword arguments they had to take for all the functionality that they were supposed to handle. Fortunately, now we have class-based views. Django registration now uses class-based views. So this is lesson number one. If something is painful, look at what Django is doing now
21:44
and maybe see if you can steer it in a direction that it will get better. In this case, class-based views solved a whole lot of problems. This one I was never able to solve. People really, really like having default templates in an application that they can just plug in and it works. Unfortunately, it's really, really difficult,
22:02
especially if it's something like, say, user registration where you don't know in advance even what kind of form they're going to use, what kind of fields it will collect, what kind of workflow it will have. Shipping a single set of templates that work for all of that is just impossible. Eventually, I gave up. Every once in a while, about twice a year, someone still files an issue at GitHub saying,
22:21
could you please provide default templates? And I say, I would really like to, but unfortunately, I can't. Then Django went and came up with this custom user model thing. Before this, Django registration only had to deal with one user model. And you knew what it looked like. It was jango.contrib.auth.models.user.
22:41
Now, suddenly, anybody can have pretty much any user they want. At first, Django registration just kind of, we're not gonna do that. Now, actually, it does support custom user models. That was a bit of work. If you want to go look in the code, you can find lots of places where it does get user model or foreign keys to settings.auth user model.
23:03
Support got there eventually, although it is still probably kind of buggy. The last two bug fix releases I've had to do involved custom users. Here's the real killer. This is two views of my GitHub contribution graph. One of them is what you see if you're logged in as me and can see all the private repositories
23:21
that I do my daily work in. The other is what you see if you're a random person who's not me looking at my GitHub profile. As you can tell from this, about two thirds of all the coding I do happens in private repositories. This means I have very little time. So, for a couple of years, I kind of dropped the ball on Django registration.
23:41
Some people forked it, tried to keep it alive. Thankfully, now I'm back with some time and able to do this. But it's a thing that happens. Don't be ashamed of it. It will happen to you. Django registration has grown some new things which you can read about in the release notes. Activation with HMAC, reserve name validator, disallowing sketchy unicodes.
24:01
If you want to hear about sketchy unicode, come talk to me or Katie McLaughlin. And there will be a future. Alrighty, so I'm here to talk about how flexibility saves you money and it gets you diversity for less than free.
24:26
Alrighty, so the old way that things work, the old HR model, if you think about a big company, is that someone's born, then they get educated,
24:40
then they go and work for 40 years, and then they retire, and then finally they don't live anymore. The thing is that that doesn't really work in the modern era because there's things that happen. People have kids, families have needs. Say you might have a parent who needs help
25:02
and you're a caregiver. So the old model of just being born, educating, work, and retiring, it just doesn't really sort of fit the mold. So only a minority of people really can do that. Now the reason this has diversity implications is this impacts women more than men,
25:23
but it's impacting men more than it used to. So this affects all of us. And the answer is we need flexible workplaces. And I want to think about, I want to just quickly go through two examples.
25:41
One is a large Australian bank where the CEO a few years ago said that they were going to have flexibility in every position, and they were going to find flexibility when people thought there was no flexibility. So it's like some jobs they could work from home, but if you're a bank teller, you can't really work from home, you need to be your customer facing.
26:02
But they went through and say, well, if someone's doing that work, how much they actually customer facing, how much like if you've got 10 people that are in the branch, maybe only two or three are actually customer facing at one point in time. So there is still opportunity for flexibility.
26:21
And so when they went about doing this, and they rolled this throughout the whole organization over several years, they found that their metrics on attracting and retaining women significantly increased. Many of their staff that would go off on family leave would find that the progression back to work was far more smooth,
26:42
and also worked much better when they're having young children and the little simple things like being able to move in a different office so they could pick up their kids from sport started to really come in into play. The bank sees it as a key part of the competitive advantage,
27:02
because other employers are just demanding that you work nine to five or eight to whatever, and or if you're IT, then seven to 10 p.m. And which unfortunately, that's really a myth for sustainable productivity.
27:20
So that's a big example. I'll give a few personal examples from the organization that I'm involved in. One is we've got one of our people involved is someone who's very skilled. She's got a few kids. She can't work five days a week. She doesn't want to work five days a week. She certainly doesn't want to work nine to five.
27:40
She's got to pick up the kids from school. Her husband is a teacher, and he can't pick up, at a different school, and he can't pick up the kids. And she doesn't want to use after-school childcare. And I'm like, well, that's all fine. We're working collaboratively on our project,
28:01
and I don't need you to be present at particular times. And she's like, well, that's fantastic. That just fits my needs exactly what I want. The second example is someone who's also very skilled and got a few university degrees, but she's developed a condition where she can only really work about eight hours a week.
28:20
But they're the really productive eight hours. She cannot, she's on the disability support pension, she can't get traditional work at all. But she's a very valuable member of our team. So the core message in all of this, if you're involved in a startup or you're in some position where you need to make these kind of hiring decisions,
28:43
then what are you gonna miss out on if you're not flexible in the way that you go about employing people? Thank you. All righty, it would have really sucked if I couldn't have gotten the AV to work after the running around I've been doing.
29:00
But I'm gonna talk to you about introspective RESTful APIs with Postgres and Django and how it will make your entire life better, but not really. So words is where I work. That's short for Word and Research Data Services. It's a single source for leading global research databases. So we have a lot of data. We have over 450 institutions worldwide who subscribe to our product. And 50% of that growth has been international
29:22
over the past five years. So the upshot of this is we have 50,000 active users who access these data by web, SSH, remote connects. And we support SSH via the cloud, SAS, R, Python, the web, and more. We have 350 raw terabytes of data
29:40
and two petabytes of total storage that we've gathered since 1992. And we're continuing expansion beyond financial data. When I started, everything ran on a single Sun E20,000 before we maxed out the RAM CPU and pegged the CPU at 100% capacity. In 2009, we switched over to a sea of white Linux boxes backed by NetApp.
30:02
We've used Sun Grid Engine, Oracle Grid Engine, Unova Grid Engine, and LDAP to support that horizontal scalability. Permissions and ACLs are always a big concern for us because our 450 customers each have access to different financial data sets from different vendors. So you can do the math yourself on how many different permutations that is.
30:21
We also made some very dubious database decisions. Spoiler alert, Postgres has made everything better. All data was stored in SAS data format in a cryptic format called SAS7BDAT. We had MySQL for website data. We then switched to SQL Server for website data. We then had research data
30:40
except for the New York Stock Exchange Trades and Quotes database in Oracle. And we still had this problem with permissions. So Postgres to the rescue. There was apprehension by the heads of our division that we were throwing yet another database into the mix while promising once again that this would be the one all be all solution.
31:01
And they were concerned that yes, we were going to now have Postgres in addition to MySQL, SQL Server, SAS. If you include LDAP, BDB, and rightly so. But all research data was successfully loaded into Postgres. First time we'd ever been able to do that. SQL Server failed miserably. Oracle failed miserably. MySQL we didn't even try.
31:25
Postgres is now backing Django for the website as well. Django and Wagtail are now in our mix instead of ColdFusion, PHP, and WordPress. We have integration with LDAP and permissions ACLs. And a goal we're still working on
31:42
is making Postgres the canonical source for all of these ACLs we need to apply to all of our data. We'd promised we'd eliminate at least one of these databases within six months. And we've actually done two. Goodbye Oracle, goodbye MySQL. And we've actually gotten
32:00
the entire New York Stock Exchange trades and quotes databases that's growing by 1.5 terabytes a month into Postgres. It's a very narrow database but a very tall database because you've got to imagine every bid, trade, and quote going across the NYSE. It's a hell of a lot of data. We have shuttered and closed our WordPress satellite site
32:22
in favor of Wagtail, which if you haven't used it, it's the only CMS I've used in my life that I haven't absolutely hated after a year and actually still love. So I've got to give a shout out to Torchbox because it's absolutely lovely. So we still have some SQL Server around but that's gonna be the next one mostly on the way out but some of our data vendors have it as a dependency
32:41
so it may never go. So foundation complete. Now we wanted to build a RESTful API because we can let people access directly through R, SAS, various other ways. But we never had a way that we could easily build a RESTful API. And we have over 20,000 basic data sets and tables available in Postgres now.
33:03
So when words was started in 1992, it was made to dump data to Lotus 1-2-3. So brilliant financial faculty around the world could use a tool they knew how to use, Lotus 1-2-3 at the time. And in 2017, we created a RESTful API to anywhere because a Fortune 50 client of ours, most of our clients are business schools
33:20
but we've got some corporate clients really wanted to be able to hit a RESTful endpoint. So we used Django REST framework. We automatically introspected Postgres to create serializers, filters, models and permissions. Code generated by introspecting Postgres SQL and we have web browsable endpoints for the user. Filters were created for any columns
33:41
that were indexed in Postgres SQL automatically. So let's do a quick, quick, quick, quick demo. Okay, we're going through the code there. We've gone through the code. And right here, here's the code to call the actual endpoint using IBM's QSIP.
34:03
And right here, we're running against the actual endpoint and we're gonna get every end-of-month data price for IBM since 1925 with the volume and the price. And this is what 20,000 endpoints looks like.
34:22
Thank you. Thank you, Tim. From UC Irvine. It's a very short lightning talk for me. It's my first time, so I'm first attendee here.
34:41
Well, I'm here because I want to find a job so I will graduate this month and they say yesterday, I had a picnic in that place and thanks for Steve, Chase, Peter, and Tristan. Yeah, like encouraged me to stay here. So I'm looking for a job here.
35:05
So, so why should, so why you should hire me because I'm self-motivated. I'm willing to learn on good habits of writing code. Most important, I'm just Django. Yeah, hire me, please.
35:23
And lastly, thank you. Okay, so I wanted to just give a quick talk about keeping bees because it came up at breakfast this morning. I am actually not a beekeeper. My wife is a beekeeper.
35:41
She's awesome at running an apiary that we have at our house. But because of all of the things that I've learned from her, I thought it would be cool to share some of them with you. So this first picture I've got, all of the photos are from the bees that she keeps. You'll see the kind of classic comb shape.
36:00
It's not all honeycomb. Some of it is brood comb and my understanding is it's shaped, they extrude the wax and they stamp it out and so the shape of that classic hexagon shape is from the shape of their head as they press the wax. If you want to keep bees, you'll need some boxes.
36:21
So there are brood boxes and the typical way of keeping bees is that those boxes are filled with frames. So there's usually 10 frames in a box and the bees will kind of start building out their comb, whether that's brood comb or honeycomb, in those frames and so that's a photo of some of the boxes
36:42
and frames that we have. If you want to get those boxes filled with bees, you can actually buy packaged bees. They come in these little boxes you see in the front of the hives there and I think there's about 10,000 in there and there's a metal can that has sugar syrup so that they have something to eat in transit
37:02
and you just pop the lid off and then you take the can out and then you pour the bees into the box. It's the craziest thing, like pouring bees. I've never seen that before but you just dump the 10,000 bees in there. If you don't have packaged bees, you can collect a swarm. When you see a bunch of bees out, they're actually really docile at that point
37:22
and they all come together and you can usually just cut the branch off that they're hanging on and then you set them on top of an empty box and they just kind of melt down into the box and they're like, ah, I'm at home. So it's really fun to watch them do that. This is a photo of some bees that are on the front of the box. That's called the landing board
37:41
because they fly in, they land right there and if you look really closely, you can see the pollen baskets that they have like on those back legs. They stuff them full of pollen and they bring it back. The pollen is actually protein so they eat pollen besides nectar that they'll gather and so you'll also notice there's that board, there's that kind of like white strip there.
38:01
There's an entrance reducer that you can put on the hive once you first load them in there. This helps them defend their hive so normally they have a nice big place on that landing board. They can come and go but at certain times of year, you put the entrance reducer on there and it helps them manage their hive so they keep pests out and stuff like that. This is what a brood frame looks like.
38:22
So the queen lays eggs into the comb and then they cap it and then you have bees that maintain it and you can actually just lift these frames right out. It's super cool. If you ever have an opportunity to go to an APA or somebody that keeps bees, ask them to crack the box open and pull a frame out and check it out.
38:41
It's really neat. A lot of times my wife will use, she has like a beekeeping suit and all that stuff but sometimes I'll just go out there with her and you just go use your bare hands. It's usually pretty cool. And one of the interesting things is the bees actually glue these frames in. They have something called propolis. That's like super glue.
39:00
It's ridiculously strong and hard and they will glue things together. Like they glue the boxes together, they glue the frames down and so you'll have like a beekeeping tool. I didn't have a photo of one but it's this metal kind of like knife looking thing that beekeepers will use to crack the boxes open and break that propolis. But you can actually scrape the propolis out and then melt it down
39:20
and then use it to do all kinds of different things. So for instance, like if you had like a bad cut on your finger or something, you can use it just like you would super glue instead of getting stitches. You can just glue it shut with propolis. It's crazy. It's ridiculously hard, strong stuff. Bees also do something that's interesting called bearding.
39:41
When it gets hot, they actually gather on the outside of the box and so they will cover the outside of the hives and they actually look like they're dripping down like a big beard. This photo kind of shows a bunch on the outside. They're not like dripping. But another interesting fact is they don't fly at night. So in the evening, they all kind of just walk around
40:00
and they don't do any flying. And so that's just something I thought was interesting. And the waggle dance, this is also something interesting that bees do. They actually do this, they spin and vibrate and that kind of tells other bees where flowers and sources of nectar are located. And it's so well known that you can actually set up a camera, watch the bees and then calculate where those sources of flowers
40:23
and stuff are at. And you could like fly a drone out there and go look at where the bees are headed. And when they come back from getting the nectar, they produce honey and then you can extract it out of the frames. And this is what it looks like when you're extracting. I'm really proud of that.
40:40
Thank you.