Building a Memex in Ember
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 27 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/35706 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer | ||
| Production Year | 2018 |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EmberConf 20188 / 27
3
4
6
8
12
13
15
16
18
21
22
23
25
26
00:00
VideoconferencingBuildingLinker (computing)MedianComputer programEnvelope (mathematics)Open setClosed setSoftwareXMLComputer animation
00:42
Projective planeTouchscreenMultiplication signExpert systemKeyboard shortcutConnectivity (graph theory)VarianceReal numberLiquidQuery languageDigital photography
01:40
Digital photographyAnalog computerCalculationNeuroinformatikAnalogyComputerMechanism designComputer animation
02:16
NeuroinformatikCalculationCalculusAreaINTEGRALData managementProcess (computing)Reading (process)
02:47
Spring (hydrology)InformationLevel (video gaming)MassMultiplication signPhysicalismComputer animation
03:19
EvoluteInformationForm (programming)Process (computing)WordTraffic reportingComputer animationMeeting/Interview
03:56
Library (computing)Product (business)Information overloadInformationPoint (geometry)Library (computing)Traffic reportingGraph (mathematics)Dynamical systemMessage passingCorrespondence (mathematics)Computer animation
04:44
Alpha (investment)Cache (computing)Computer fileNumerical digitRule of inferenceInformationElectronic visual displayLinear codeGraph (mathematics)CodeTouchscreenPunched cardCodeRight anglePeripheralLink (knot theory)Key (cryptography)NeuroinformatikMultiplication signDecision theoryDisk read-and-write headMathematicsAssociative propertySubject indexingEngineering drawingComputer animation
05:49
NumberTouchscreenInformationSoftware developerInformationGradientLimit (category theory)TouchscreenCodeBitComputer animationLecture/Conference
06:26
ChainMenu (computing)Line (geometry)Control flowConvex hullInterior (topology)Scalable Coherent InterfaceTraffic reportingPlastikkarteDigital photographyComputer animation
06:56
BlogLoginService (economics)Data conversionSampling (statistics)Game controllerSlide rule
07:47
Source codeWeb browserTwitterVideoconferencingGraph (mathematics)Message passingLink (knot theory)Physical systemQuery languageSoftware developerReading (process)MereologyGraph (mathematics)TouchscreenMultiplication signWeb browserE-bookService (economics)Keyboard shortcutStructural loadProjective planeQuery languageDemo (music)Message passingSet (mathematics)Physical systemInteractive televisionUniform resource locatorFront and back endsPattern languageForm (programming)Transformation (genetics)Digital photographyCartesian coordinate systemTwitterMoment (mathematics)Workstation <Musikinstrument>Theory of relativityBuildingSinc functionOpen setAdditionMusical ensembleComputer animation
10:05
Query languageView (database)Directed graphQuery languageSoftware repositoryGraph (mathematics)WeightRepository (publishing)Multiplication signVariety (linguistics)Different (Kate Ryan album)Numbering schemeLiquidComputer animation
11:02
Keyboard shortcutClient (computing)Revision controlAliasingQuery languageType theoryTraverse (surveying)Graph (mathematics)Different (Kate Ryan album)Query languageMultiplication signGreatest elementPredicate (grammar)Reading (process)Object (grammar)Cartesian coordinate systemSign (mathematics)Computer animation
11:49
Query languageBuildingMathematical singularityComputer iconSystem callCybersexQuery languageResultantSubgraphQuicksortGraph (mathematics)Computer animation
12:31
GoogolQuery languageMultiplication signPattern languageBitMessage passingComputer animation
13:07
MultilaterationBitMessage passingEuler anglesContext awarenessCurvatureMultiplication signMathematicsTouchscreenData conversionInternet service providerComputer animation
14:03
Token ringDigital photographyCartesian coordinate systemRoutingDimensional analysisMedical imagingTouchscreenBitSoftware developerEvent horizonComputer animation
15:06
Query languageCodeStack (abstract data type)Data conversionGoogolTerm (mathematics)Multiplication signLine (geometry)System callQuery languageLevel (video gaming)FlagTime travelResultantStack (abstract data type)BitBuffer overflowSelf-organizationQuicksortComputer animation
16:12
Data conversionStack (abstract data type)CodeQuery languageGoogolModul <Datentyp>DrehfeldMultiplication signDifferent (Kate Ryan album)Type theoryNumberService (economics)Right angleMereologyProcess (computing)Marginal distributionPoint (geometry)VotingQuery languageDigital photographyComputer animation
17:17
Limit (category theory)Complete metric spaceElectronic visual displayStructural loadReading (process)TimestampMereologyCodeComputer animation
17:58
Query languageFingerprintView (database)Reading (process)Filter <Stochastik>Web browserData structureAssociative propertyThread (computing)Computer animation
18:29
Reading (process)Computer animation
19:00
Duality (mathematics)DialectHacker (term)WikiQuery languageReading (process)Context awarenessReverse engineeringTraverse (surveying)Link (knot theory)WordMultiplication signSimilarity (geometry)Query languageComputer animation
19:43
Musical ensembleCodeBuildingBlogWebsiteQuery languageOvalTimestampAnalogyBitInheritance (object-oriented programming)Digital photographyMultiplication signMereologyContext awarenessFamily1 (number)CASE <Informatik>Cloud computingSquare numberSimulationRange (statistics)Form (programming)SpacetimeMedical imagingLink (knot theory)Line (geometry)Computer animation
21:50
FingerprintInclusion mapTerm (mathematics)Row (database)QuantumLevel (video gaming)Semiconductor memoryMultiplication signFlow separationLoop (music)Computer animation
22:48
Computer wormQuery languageDigital photographyDevice driverExpert systemMatching (graph theory)Heegaard splittingMathematical analysisComputer animation
23:55
Query languageNetwork topologyService (economics)WeightRing (mathematics)Context awarenessMereologyMusical ensembleParticle systemOffice suiteLevel (video gaming)QuicksortComputer animation
24:39
Query languageContext awarenessRow (database)Digital photographyHand fanVideo gameMultiplication signMusical ensembleMappingRoundness (object)GodComputer animation
25:37
Maxima and minimaExecution unitLevel (video gaming)Slide ruleMultiplication signInformationCodeTrailDemo (music)Context awarenessComputer animation
26:45
Demo (music)ParsingBitChainPoint (geometry)Demo (music)Stability theoryCASE <Informatik>Multiplication signVideoconferencingTrailPresentation of a groupVideo gameProjective planeBus (computing)Goodness of fitPublic key certificate
27:41
Connectivity (graph theory)Modul <Datentyp>Graph (mathematics)Skeleton (computer programming)Function (mathematics)Element (mathematics)Single-precision floating-point formatState observerGraph theoryGraph (mathematics)Connectivity (graph theory)Functional (mathematics)Multiplication signSystem callMathematicsTime seriesCross-correlationComputer animationLecture/Conference
28:13
Graph (mathematics)Computer networkRotationLink (knot theory)Connectivity (graph theory)Graph (mathematics)Cross-correlationGraph theoryGrass (card game)CASE <Informatik>Graph (mathematics)SoftwareNumberEndliche ModelltheorieType theoryMappingCountingCategory of beingTraverse (surveying)IterationMathematicsGraph coloringCycle (graph theory)Right angleComputer animation
29:13
BuildingProcess (computing)CuboidMappingBuildingType theoryDemo (music)CodeQuery languageMultiplication signBit rateComputer animation
30:06
WindowState observerComputer configurationExpert systemDemo (music)State observerFrame problemLine (geometry)Mobile appTelecommunicationComputer animation
30:37
Extension (kinesiology)Computer configurationProcess (computing)Data managementStructural loadElectronic visual displaySource codeComputer-generated imageryProcess (computing)View (database)Interior (topology)Web 2.0Mobile appPhysical systemFront and back endsWindowTelecommunicationConfiguration spaceInstallation artComputer animation
31:07
DatabaseBackupBroadcast programmingPoint cloudConfiguration spacePhysical systemEndliche ModelltheorieNumberOpen sourceTelecommunicationProjective planeDirection (geometry)WordComputer animationSource code
31:42
NumberGraph (mathematics)Level (video gaming)Multiplication signType theoryVideo gameNumberRevision controlAugmented realityComputer virusComputer animation
32:14
Directed graphSoftware testingVideo gameData structureOpen sourceBridging (networking)DatabaseRight angleProjective planeType theoryInverse elementMoment (mathematics)Computer animation
32:46
DivisorComputerInternetworkingProjective planeType theoryFamilyNeuroinformatikForm factor (electronics)Hard disk driveSheaf (mathematics)Sign (mathematics)Near-ringComputer animationDrawing
33:29
Smith chartMultiplication sign1 (number)NeuroinformatikProcess (computing)Goodness of fitCodeAnalogyDigital electronicsSocial classAnalog computerCalculationElectronic mailing listOnline helpComputer animation
34:11
BitPerturbation theoryAnalogyMultiplication signTube (container)Physical systemNeuroinformatikVacuumBlogComputerComputer animationEngineering drawing
34:50
Serial portVideo gameFormal languageVirtual machineElement (mathematics)Uniqueness quantificationKey (cryptography)VibrationStructural loadType theoryWage labourTable (information)Content (media)OvalVideo gameSuite (music)Cellular automatonLine (geometry)Computer animation
35:30
DivisorProjective planeMultiplication signProcess (computing)Execution unitWhiteboardComputer animation
36:03
Metric systemProjective planeMultiplication signNetwork topologyVector potentialMedical imagingComputer animation
36:39
Perturbation theoryComputer networkGreatest elementElectronic mailing listWindowKeyboard shortcutPhysical systemNeuroinformatikGame controllerBuildingBerners-Lee, TimMathematical analysisEngineering drawing
37:09
Projective planeWeb 2.0Product (business)Dependent and independent variablesSoftware developerMultiplication signPattern languageBoss CorporationBitInclusion mapACIDTwitterComputer animationMeeting/Interview
37:48
Projective planeMultiplication signoutputDigital photographyData conversionSemiconductor memoryLocal area networkPlanningBitComputer animation
38:22
Revision controlNewsletterLinker (computing)AverageRSA (algorithm)Projective planeComputer animation
38:52
Coma BerenicesBlock (periodic table)Data typeXMLComputer animation
Transcript: English(auto-generated)
00:02
So, I'm Andrew Lewis, I'm from Toronto, Canada.
00:23
They decided to put the Canadians together, so welcome to EmberConf Canada 2018, Godfrey's up next. So I'm the opening keynote for EmberConf Canada, Godfrey's the closing keynote. That's our mini program for the next two hours. So I'm Canadian, I'm from Toronto, I'm a software developer, but I'm gonna start with a confession.
00:43
I have no idea sometimes what I'm doing with Ember, sometimes I'm looking, I really don't know what I'm doing. I'm not an expert, I'm not here to talk to you about glimmer performance, or I'm not gonna make dancing Tomsters move around the screen in beautiful fashion. I'm just gonna talk about a side project that I'm working on that happens to involve Ember.
01:00
But first, I'm gonna talk about history. So in the tech world, we come to a conference like this, we hear about history, and we have this funny idea of what it means. So this is history, we don't use var anymore, we use let. We don't use import ember anymore, that's history, we use our components. If you were in the keynote yesterday,
01:21
in the talk yesterday, you would learn that we don't use liquid bind anymore, that's history too, we use animated value. And then if we go back into the ancient history, we got jQuery, this is ancient history, and then even stepping back further to prehistory, we have the on clicks. So in the tech world, we have this funny idea of history. I'm gonna talk about real history,
01:40
the kind with black and white photos. And I'm gonna start by telling a story, and the story starts in the 1930s. And it involves this guy, Vannevar Bush, and if you've heard the name before, and you're talking about Vannevar Bush, and you hear someone saying that, you can say, well actually, it's pronounced Van-ee-var. I don't know why, but that's how you pronounce it. So Vannevar Bush was an engineer,
02:02
he worked in the 30s, and he built the first analog computers, and these were cool devices, and they would do a calculation, and they ran mechanically. There were no digital computers back then, and he was a really good engineer, he built a bunch of these in MIT. So here's his most famous one, the differential analyzer. And what you do is you trace a curve,
02:21
the person on the right side is tracing the curve, and the area underneath would be calculated by the tracing of the curve. So these are incredibly cool computers, they were used for ballistic calculations, and for integration and calculus, and they were really cool. Like a lot of engineers, and they're approaching the end of their careers, he got pressed into management.
02:42
So his job turned from making cool devices into reading documents. And in the 1940s, something big happened, that was World War II. And pretty much the whole country reoriented towards winning the war effort. And Vannevar Bush, he was called the general of physics, because his role was to give science advice
03:01
to the president. And he had a lot of really big contributions to the war efforts, he made it to Time Magazine. But one of the big stories about World War II was that there was an explosion in information. So there was all these tools for creating information, but during the World War, there was this, the first time they got used at a mass level. So there were new bureaucracies,
03:20
they were producing endless amounts of data, new data, and everybody who wasn't on the war front was somehow producing data in the form of a paper and information back home. So here's some just examples of how much information was being produced, it was unprecedented. And Vannevar Bush's job was to just deal with reports, and he'd get science reports, and he'd have to read them and understand them,
03:42
and then translate that into advice for the president. So across his desk, there were 35,000 reports that came to him during the course of the war. So it was just an overwhelming amount of data. And he was very frustrated by it, he felt like not in control, and he had a phrase, he was being buried under our own product. So we have all these technologies
04:01
for creating information, but we don't have technologies for actually processing and understanding it. So when the war came to an end, he thought about it, he thought about solutions, he put his engineering hat back on, and he wrote an essay called As We May Think. Short little essay, it's pretty easy to read, it was published in July 1945. And in it, he describes a device that could solve this problem for him,
04:21
and he called it the Memex. So what is the Memex? The idea was that you'd have a device, a desk-sized device, and you put all your library in it, you put your correspondence, your messages, your reports, and the point was that you'd be able to search it really fast. So this was like one device for all the information that you'd ever come across and understand and see,
04:40
and you'd be able to really stay on top of information overload. It was a pretty simple device, not much to it, it was desk-sized. It had a couple of displays, it had a little punch code for putting in your codes, and it had microfilm, and that was the way information was stored. And really the idea was like, everything could be put on microfilm, so you could hold a lot of information.
05:01
This is a close-up of the screen, and so on the left, it would display the information from the microfilm. On the right, you could add your own notes. It came with this add-on peripheral for creating voice memos. It came with another add-on peripheral. This is a camera that you would strap to your head. Cyclops camera, they called it. And all this was just different ways of getting information into your Memex.
05:21
But the key design decision of the Memex was to use, instead of having alphabetical or topical indexes, Vannevar Bush really wanted to have a different way of organizing information, so he decided to use links. And this was really one of the first times that links were used in this way. So the idea was, if brains work this way, by association, we navigate by association,
05:43
you thought computers should be able to work this way too. So it was really cool, it was very forward-looking, but the unfortunate thing about the Memex is that it was never produced. So this made me sad, very sad. And you know, as a good developer, I said, I should just build my own Memex. This would be great.
06:02
So I said this naively, and many, many years ago. So I'm just gonna talk about that journey now. But first, I'm gonna talk a little bit about my psychology and I promise there will be code on the screen eventually, if you're seeing too much text. So the problem I want to talk about, it's about a problem about myself. I'm an information pack rat.
06:21
So I save everything and I have a real emotional problem with letting go of information, I can't do it. So in grade five, I started keeping a journal because I didn't want to forget what happened each day. I saved all my kindergarten report cards. I have every movie stub for every movie I've ever been to and kept them all. I have every photo that I've ever taken to thousands over the years.
06:41
Pre-Google Maps, I even tried to map my walk, so this is like my walk through Toronto. And it didn't work very well, it only lasted about a week, this experiment, but I tried. I have good notes from almost every talk I've been to and this is pretty useful. I have all my conference badges, obviously. Worst of all, I saved all my MSN logs from high school.
07:03
And obviously, there's really not much that's worth ever reading. This is a sample conversation, which quite clearly, I really shouldn't read this very often, it's awful stuff. When I'm such a nostalgic person, that I even had nostalgia, I had preemptive nostalgia.
07:20
So here's a chat log where I'm chatting with a friend and I remind her that I have logs. I tell her how to do it. And then I suggest that in 10 years, we should get together and read these logs. So it's very nostalgic in advance. And then at the end, she calls me a freak because I think, yeah, this may be a little bit weird. But this is the kind of person I am. I can't help it. But the problem is that we have
07:40
all these digital services now and the problem has skyrocketed. It's out of control for me. So I have an overwhelming amount of personal data to save. So there's a couple solutions. I could talk to a therapist. But I decided not to do that and instead, I used Ember.
08:02
I was a good developer. All right, so I'm gonna talk now about building a memex in Ember. So the hardest part of this project has been just gathering the data. It hasn't really been a front-end problem. It's been a data problem.
08:21
So I've built importers from almost everything I can get my hands on. So I have all my reading and browsing activity for almost 10 years. My RSS reader, which I still use, which is great. Twitter, ebooks. I have all my digital consumption. So every time I watch a movie or I like something on Instagram or a podcast listening history, I have in there. And these are coming from all the services I use.
08:41
I have location history from my phone, which is a really good beacon in my pocket. I have all my messages for almost 20 years of messaging history and social interactions. And then there's a lot of qualitative data in the form of journaling, which I've kept on and off for the last 20 years. I have my photos, I have my notes. So there's a really rich set of history that I have.
09:02
So it's from almost 1995 onwards. That was my first journal, and I have almost something for every day since then. And it's organized into a graph that I can navigate it, and it's a pretty loose schema, but it allows me to really navigate it like the mimics. It has about five million activities of things I've done, and then four million entities,
09:21
and these are people, places, things, concepts. And then there's a lot of relationships between them. They bind them all together. And then there's an ETL system. ETL stands for Extract, Transform, Load. So it's a pattern for having an importer and then transforming that, and then loading it into my mimics. And then I have a custom query language
09:41
for actually querying it, and then I use JSON API in the backend. So now we come to the scary time, the live demo. If you all wanna bow your head and give a little prayer, that would be appreciated. And I'm just gonna switch to my browser. I hope everybody's praying.
10:02
Cool, okay, so this is the opening screen. So on the left is our query builder. So right now I have a simple query. It's just everything I've done on GitHub. So you can see, you know, this is a lot of all kinds of different activities, and it's a lot of time. I can scope it down to something like verb liked. So this is all the repositories I've liked,
10:22
you know, a lot of Ember stuff. And to give a better idea of how the scheme is organized, I have this node view, and you can kind of get an idea. So in the middle is the Andrew Lewis node, because generally it's oriented around things I've done. And then coming out of it are the repos. So this is Ember liquid sauce. These are the owners of the repos,
10:42
and then this is like the tags. So you can kind of get a sense of how everything's organized as a graph. And we navigate this graph with the custom query language via this builder in the left hand side. So I can change it to verb edited. So this is what I use for commits.
11:00
This is the verb I use for commits, and it updates. And so I'm just changing the traversal. So I'll show you a different type of traversal. Let me turn off the graph. So I'm gonna change it to verb listened. And then, so involved, every activity has an involvement. So this is like, if you think of subject predicate object, like this is the object.
11:21
So I'm gonna look for songs. And I'm gonna look for songs where the creators are Aretha Franklin. So what we have here is a three hop traversal where we evaluate from the bottom. We find all the nodes that match Aretha Franklin. Then we find all the songs where she created them, and then we find all the times I listened.
11:40
So let's evaluate this query. And I might have messed something up. Oh, songs, yeah, I need a better autocorrect. Cool, so this is every song I've listened to by Aretha Franklin. So it's a pretty simple traversal. I can add full text search into here.
12:01
So let's say love, so any song that matches love with the creators of Aretha Franklin. And we just update, and we get that. So that's an idea of how we build a subgraph. We describe a subgraph that we want, and then we evaluate the query, and then we see the results on the timeline. So just to give an idea of how much is in this thing,
12:21
I'll do a pretty simple query for something like Christmas, which I don't really care much for Christmas, but it goes across all my data. So let's evaluate this query and graph it. So you can see there's a lot of hits over time. So from 2002 onwards, you can kind of see
12:40
the seasonal pattern of stuff. So let's do something a little bit more useful than Christmas. Let's search for ember. So this is everything I've done that involves ember over quite a few years. And we can scope it down to verb messaged. So this is every time I've messaged,
13:01
and it's had ember in it. So let's explore it a little bit. So this looks pretty early on. I'm just gonna have to refresh, because I messed up. So this is pretty early on. I think this is too early, 2010. This is a message where my friend jokes about the embers. It's not the real embers. So let's go a bit earlier, a bit later. So this is a message from 2012.
13:22
My friend Phil is complaining about learning ember. So for any of these items, we can always find the context and what was going on around that time. So let's have a look what else he was saying. I asked if it was heavy-handed. He says it's kind of like Rails. He complains about knowing JS, and he says it's all magic.
13:41
So this is like an early conversation. I can find it very easily. By the way, I asked Phil if I could show this on screen. He said it's fine. He says he loves you all, and he's changed his mind. So just wanna. I'm not just compromising him right now. Okay, so what we can do next, we can change this around a bit.
14:01
I can add a provider, so Slack. So this is like everything on Slack that I mentioned. If you notice, it's pretty sparse. That's because the ember community Slack doesn't allow you to have an access token. So if there's anybody in the room who has admin access and wants to give me an access token, that I can get my data and my Memex, I would really appreciate it. I don't know who I can talk to after about that.
14:21
I can also do something like this, where I change the verb to photographed. And this is a bit of a weird query, because if you'll see, there's really nothing in the photo that suggests it might be ember, but we have OCR on the photos. And yeah, so here we have Matia doing a meetup,
14:40
and we have ember on the screen. So we were able to match for the full text that's in the image that I took of the event. Thanks. Yeah, so this is actually surprisingly useful, because Twitter, we have these screenshots of text, and we all favorite them because we agree with them, and then we can never find them. So I don't really have that problem.
15:00
I can search within these awful screenshots of text. So it's pretty useful. As a developer, there's another thing that's useful. I have all my command line history. So I can say verb commanded. So there's a tool I use called GIFsicle for making GIFs, GIFs, GIFs. And I can never remember the flags. I think this is a common problem. And I only use this thing a couple times every year,
15:22
so obviously I don't remember the flags. It's pretty easy to get to all the times I've used them. So let's do a look. So this is a query for every command that involved GIFsicle. So here we go. It's kind of like the play-by-play, if you will. And this is maybe as useful. Sometimes we actually want to learn a little bit
15:41
about what we were doing around the time. And the cool thing about the way I have this organized is that I can always use one of these search results to jump back in time. So let's see with this one. I use the flags. I use the convert tool. Maybe that's what I was actually after. So I can load the next activities, and I can see the Stack Overflow articles I was reading. I can see all the documentation I was reading.
16:02
And when you see everything in line and collated on the same timeline, it's almost like a time machine where your brain gets the same stack that you had when you were solving this problem earlier. We can do fun things like remove rmf.modules, and we can see all the times. Oh, sorry.
16:22
Great. You saw the time I typoed the rm, and now you can see the time I actually did. I deleted my node modules. This happens quite frequently because, as I said before, I don't really know Ember too well or JavaScript too well. So it's kind of fun. It's useful to be able to see all your history this way.
16:42
I'm gonna show some other stuff. So Instagram is a wonderful service. I think a lot of us use it. But very obnoxiously, they don't really have an export feature or a history feature. And I feel, as users, we make this service really powerful, and we don't have access to our own history. So part of what I'm doing is almost political. I feel we should have a right to our data.
17:00
So I have a copy of my Instagram history, so let me just do a query for it. The verb is published, so these are photos that I published on Instagram. And this would pull up everything I've ever saved on Instagram. I want to scope it down to stuff I've done. So the actor is Andrew Lewis. So this is my, when it loads, this is my complete Instagram history,
17:21
which is great because otherwise there's really no way to get this stuff obnoxiously. We can change the way it's displayed. You know, we use kind of a mason review. I also have all my reading activity, and I think when Vannevar Bush designed the Memex, he really had reading in mind, and I think this is a big part of it. So for the last few years, I've only read ebooks,
17:42
I think just because it's easier to extract, like, you know, things I've been reading and saving. So let me show you that. So verb read. So this is like pretty much everything I've read. So I'm reading a book called Wanderlust, and it has the timestamps. I have quotes that I can add to that, so I'm now combining filters.
18:01
This is my, the reading and quoting. So it all displays, and I can search within that, obviously. But because of the way the data structure is organized, and I can navigate through with associations like the Memex, I could do something like this, where, so I look for browser history
18:20
that involved Wikipedia. So this would be every Wikipedia article I've read. I don't know why it's not loading. Oh, as verb browser, someone needs to pair with me on this. Browsed, whoops, browsed, cool. Okay, so this is every Wikipedia article I've read.
18:41
It's cool. But I can do better, and I can say, occurred during, and verb read. So this is every Wikipedia article I've read while reading. And I think there's like a habit I have when I'm reading that ebook, I'll look up something, and like, that becomes really cool. It's almost like free notes that you have of the book you're reading. So I can pull up every article I've read while reading.
19:04
So you know, like, these are the things I looked up, and it's really cool for like prompting or remembering a book. I can always see the context around it. So yeah, like this is pretty useful stuff. I can do another one, which is, so I'm gonna do something similar where it's verb browsed, it involves.
19:23
So I'm doing a traversal again, and I'm gonna do one more traversal. So I'm looking for, I'm looking for podcasts, so I'm evaluating from the inside. I'm looking for podcast episode where the references are things I've browsed. So basically in a podcast show note, you have a bunch of links. So I can do a query for every time I've looked at a link
19:41
that has been referenced on a podcast. So yeah, so this is like, you know, I listened to a podcast that mentioned this book. I clicked on the link, and now it's in my Memex, and it's connected to that listening activity. And you know, I can browse around. I was actually, I was talking to Danielle, and I wanted to find an example of Blue Apron, but unfortunately I couldn't find any podcasts
20:00
that have linked to Blue Apron. So I don't have that, but I have lots of Squarespace entries. So I started this talk by talking about my nostalgia problem. So let me do a little bit of nostalgia here. So I'm gonna look for noted. This is the phrase for my journaling, the verb I use for my journaling. And let's put it like way back, so before 1999.
20:25
And you know, a big part of being a kid is birthday parties. So I'm gonna search for all the times in my journal. I mentioned birthday parties. So here's some old ones. This is from 1995. I had just, I backfilled my data. So I have that all in there. This is, if you read it, I was really upset.
20:42
My little brother had a birthday party, and my parents didn't let me play with the kids because I was too old. So I was really upset at it, and I read a journal entry about it. So the cool thing is my parents are a little bit like this as well, and they have timestamps on the photos they took. They had that little analog date on the camera. So I can always see it in the timeline.
21:01
So these are pictures from that day. This is the birthday party that I was jealous I couldn't play with the kids. If we scroll down, here's another entry about indoor recess, and it was raining. So there's not too much context from this time, but I wasn't really generating a lot of data. But there is data in the form of weather.
21:22
So the story checks out. You can see, so for any day I can pull the weather, and sometimes it's mostly not useful, but sometimes it gives really cool context. So in this case, it was a rain day in my journal, and the story checks out. The percentage of probability of precipitation was one.
21:41
So yeah, the story checks out. My journal is true. I'm not a simulation, I guess. Okay, I'm gonna do a few different things now. Almost done. So I like walking, so let's have a look at all my walks. So this is like a record of all my walks over many years. I can put them on a map.
22:01
So that's cool. And I like long walks in particular. So this is walks that are longer than 10 minutes. So these are all my walks longer than 10 minutes. Sometimes I like walking on beaches. So this is my long walks on Bondi Beach. So this is kind of fun.
22:22
And most of the time, I don't really care to look at exactly where I walk. But sometimes it helps jog memory. So a couple of months ago, I went to Strangeloop. Has anybody been to Strangeloop? It's a great conference. So I drove down with my friend Max. It's in St. Louis. So let's have a look. So I can do verb drove, duration, let's say.
22:41
This is a long drive, so 10 minutes long, I guess. And I'm doing it with my friend Max Fatesman. Oh, sorry, occurred with Max Fatesman. So this is the drive I took with Max.
23:03
So I tagged people manually. That's one of the only manual things I have to do, unfortunately. But when I have it, I can do these really cool things like finding all the long drives I've done with Max. So it was a really fun road trip. Along the way, we took some photos. So what I can do is verb photographed. And I'm gonna put this inside.
23:24
So let me just delete this. Occurred during, and I'll just put this again. Verb drove, duration, 10. So this is all the photos I took with Max on a long road trip. And you can kind of get a sense of what we saw. It's pretty fun.
23:41
When we are splitting the bills later on, I can use the OCR to find the fuel I paid for. So that matches. So it's kind of useful. You can kind of understand. So on our drive, we listened to some podcasts, and they're pretty cool. So I can do verb listened, involved.
24:07
Am I off the screen? Yeah, sort of, okay, almost. So these are all the podcasts I've listened to along the way, which is pretty cool, because there was one podcast, we were going through the Appalachians, and we listened to a cool podcast about the rocks of the Appalachians.
24:21
So it's cool to see it on a map, and to see it in context, because sometimes I might only remember that podcast, and I wouldn't be able to search for it, but I would know how to look for it based on where I listened to it. So that's becoming kind of useful. So another thing we did on the trip is we went to a cool country bar. We're not too into country music, but we gave it a shot. So I have all my Shazam tags.
24:43
So this is actually a pretty useful data set, surprisingly. So every time I pull up my phone and use Shazam to tag it, I have it. So let's try to remember. It was somewhere around, I think it was around here. We can update the map. And there it is. We listened to Garth Brooks. Any Garth Brooks fans in the house?
25:02
Woo! All right. Great, okay. So we listened to this cool song. I have the record of it. In this bar, let's pull up the context. So you can see it was with Max. It was in Music City Bar and Grill. I can also see some other things we were doing. So I have a photo. We were eating wings, I had a beer, some photos.
25:22
So this is like a cool journaling feature where you can almost go back in time and really see what my life was like in the past. Finally, I have a lot of food, so I track my food. This is really not as useful. But it's fun.
25:42
So I have all my burritos. This is every burrito I've ever had. I have my heat map burrito. And I can scope it down to Toronto. So this is every burrito I've had in Toronto.
26:00
So this is really one of the stupidest things I track, but it's fun and I keep doing it. But there was a time when I was eating a burrito and I was reading a book and I saved a quote from that book and I really don't remember anything beyond that burrito about how to get back to the book. So I'm just scrolling through my burrito history and jogging my mind about all the times I've had burritos.
26:21
And one pops up on the map and yeah, that looks familiar, so I can use that to go back in time. I can see the context again. This is cool, that sounds great. I was reading the book, the information. And yeah, there's the quote I saved using the burrito tracker. So sometimes it turns out to be useful, most of the time it's not, but it's fun to have this data.
26:42
Cool, I'm gonna go back to my slides. Cool, so this is the end of the demo. I was joking with Matt Beal. I had it recorded just in case it backed up and I said if I get hit by a bus,
27:01
I could use the videos I recorded. Then I realized I wouldn't really be able to do that if I got hit by a bus. But with the Memex, maybe someone else could read my life and do the presentation. So this is not too long, but I just wanna go over some of the Ember stuff I'm using. I guess the big question is why am I using Ember? I started with Rails, so I think the community
27:22
went over and it's a great community. I stay for the community, it's really fun. But I think when you have side projects, having stability is a really good feature and I don't have to rebuild my tool chain every time I wanna do some work on it. I'm running a bit low on time, so I'm gonna skip the parsing. I don't think anybody wanted to see BNFs at this point of the afternoon.
27:42
D3 is pretty cool. Has anybody ever done it with Ember? It's not too challenging. So basically what you do, there's an add-on that gives you Ember D3. Use the data insert element to set up your SVG and your elements and then use an observer and every time the data changes,
28:00
you pass it into your component and you update the graph. And then you do all the weird D3 stuff in a single function or inside, outside of Ember land and it works pretty well. So I have some cool graphs. This is the time series I had. This is also just like a correlation graph. I can also do these kind of cool graphs. These are network graphs, so they show nodes and edges.
28:22
So in the case of the one I'm using, I have a bunch of Ember models and they all have relationships to each other through Ember. So in my Ember model, I have all the types of relationships enumerated. And then what I do is I use each relationship iterator to go over each type of relationship
28:43
that the node is connected to and then I recurse. So basically I'm doing almost like a recursive traversal through my Ember models. And then the tricky thing here is you have to remember that you don't want cycles, which I do have, so you have hops remaining to just make sure you never get cycles. And then the other thing I do is count how many edges
29:02
I have connected to each node and then change the property and then make the size dependent on that. So this is maps. Maps are pretty simple. I think we've all done them. What I would recommend is checking out Mapbox because it's WebGL based, so it lets you do really cool, very nice heat maps.
29:20
So check it out. So building the timeline, I'll talk about this really quickly. So I have this concept of nesting. So what I do is I have a query and it returns a bunch of activities. And then for each activity, I find the unique dates and I create another type of timeline item. And I also create the start markers. So these are like the started red. And then I put them all on the same timeline.
29:41
And then unfortunately, I have to use jQuery to actually find the height of the bars and I do that after. So everything's kind of nested in this structure. And as I scroll, these things are updated. If you see on the top left, this was supposed to work, but I started editing the code last night and I broke it, so it didn't work in the demo. But in the top left, it'll always show you hovering,
30:02
if you hover over it, it'll show you where you were and who you were with. So that depends on knowing what's visible. So this used to be really hard. We used to be able to do window.scroll. This wasn't really good. And then we used requestAnimationFrame. And now we have intersection observers. And they're really cool.
30:20
You should check them out. In the future, I'm not sure what's coming down the line, but there must be something cool. And if you don't want to learn any of this stuff, Ember and Viewport is a great add-on for doing this. So I'm trying to wrap this up into an app that people can use. So I'm putting it in Electron. So Ember Electron is a great add-on. So what I do is I have a main JS process.
30:41
This is what Electron gives you. And this runs an Ember app. So this is like an outer Ember app, which manages the windows, which manages the tabs. And inside this, there's an array of web views. And each web view has another copy of the Ember front-end app that's running. And then underneath that is Docker. So it's kind of like a four-level deep system.
31:02
But I have a little installer here which sets up the Docker. I have this configuration panel. And I have this configuration for all the importers. This is actually an Ember model. So and it serializes to YAML. And then the Docker reads that. And this is my tab system. And if Ember Electron is confusing to you,
31:21
just copy the Ghost repo. It's really cool. It's all open source. And they've done a lot with Ember Electron. So where's this project going? I think there's a few directions for me. There's the quantified self idea. So I could do stats. This is a graph of like my biking dependent on the weather. So you can see as the weather drops, I don't bike anymore. So I could do all these kind of
31:41
quantified self search things. Here's like another graph of my biking and visiting. This is a map of the hour of the day and the types of drinks I have over time. So it's kind of fun. I can do all these quantified self stuff. A lot of people are attracted to this. I kind of look at it like this. It's awful. It's kind of reducing your life to numbers and I like the journaling a lot more.
32:00
So I'm kind of aiming for this. And this seems as nerdy, but I'm thinking of this as almost like an augmented reality where the final version of this is like my life in a sidebar on my glasses and I can always see where I've been before what I've been doing. And there's someone I really like, his name is Douglas Engelbert. And he had this experiment where he tied a brick to a pencil and he tested how people write. And obviously when you have a brick strapped to the pencil,
32:22
you're not only writing poorly, but you're also not thinking properly. And I think the inverse is true. The tools we give ourselves almost create new types of behavior and abilities that we have. And I think this whole project is an experiment about what abilities emerge from my ability to have my life in a database. And yeah, I do hope to have it open source soon.
32:43
So to conclude, I'm gonna go back to the story of the Memex. The Memex was really weird. Some people have heard about it. It's really considered like a really early internet device. People celebrate it as this really cool type project. But it was weird. And I think a lot of people concluded this when they saw this picture. You know, it's a weird device.
33:01
But it was weird for other reasons too. The form factor was weird. So a computer in the 1930s was this. And if you're thinking, oh, these are nice little computers people are using, that's not the computer. The computer is the room of women working. Generally that was a computer in the 30s. You can see the sign here, computing section. So the form factor was weird.
33:21
When Venerable Bush created it, the idea of having a desk-sized device was really strange. Even in the 50s, this is a hard drive. In the 60s, this is a computer. So even for its time, it was very weird. Someone actually created replica Memex, and this is like what it might look like. It was weird for another reason. It was weird for the problem it solved.
33:42
So computers were typically used for big, important problems. So this is the Colossus in Bletchley Park, and it was used for cracking Nazi codes. So a very important problem. It did a good job of it. This analog computer was used for ballistic calculations, also a pretty important problem for the world during World War II. This was the UNIVAC, another early computer,
34:01
and it was used for the census. So computers were used for these very big problems. The Memex was weird because it just wanted to help an individual user stay on top of a very personal problem. It was weird for another reason. The technology is used. So if you're thinking and looking at this and thinking, wow, it's really old-fashioned. Even for its time, it was a bit weird.
34:21
It was all analog, and there was a new hotness at the time. It was called vacuum tubes, and all the cool new computers were using vacuum tubes. Vannevar Bush really didn't understand them, didn't like them, and so he designed his system almost like in tools from the last decade. So this is the first digital computer. He didn't really get it,
34:40
so he stuck with what he knew, and that's what he worked with. It was weird for another reason, how it was published. A lot of big ideas come out in journals or blog posts or big important places. He published it in the Atlantic Magazine, which is a general reader magazine. This is the table of contents. You can see alongside it, it was Forgive Me, Stranger, a Poem,
35:01
Ships and Subsidies, The Egg and Eye. So this is the company it kept in the literary world, and then it was republished again in Life Magazine. So it's kind of nestled amongst a lot of general content, and if you have a look at, this is the Memex essay with advertisements in the sidebar, one of the most important ideas in computing history
35:21
was next to a laxative ad. So this is not really where you think a big idea was launched, but this is what it was. So it's weird for all these reasons. The reason I think it's weird and exciting at the same time, it was a small little side project of Vannevar Bush. He was busy co-founding Raytheon.
35:41
He did this earlier, which is one of the biggest contractors for the US defense. He was actually also the civilian leader for the Manhattan Project. So while he was thinking about the Memex, he was actually in charge of making sure that this device got built. And then after the war, he ran the National Science Foundation, and he was on boards,
36:01
and he had many jobs. Even when he retired, he kept on tinkering with stuff. He created a lot of stuff. And the Memex was always this thing he thought of. He had ideas about and published. And he kept on publishing it into the late 50s. So this was something he thought about. It was a side project,
36:20
and he never spent all his time, but it had an incredible amount of impact on the world. So a researcher, Linda Smith, looked at all the citations that the Memex had, and she described what it had as an image of potentiality. So it was weird, it was different, and people latched onto the weirdness in their own ways. And there's hundreds of citations over the years. People like to make these trees of what the Memex influenced.
36:42
You can see at the bottom, Windows CE is on the list. So everybody likes to trace their origins to the Memex. This is Doug Engelbert. He was very inspired. This is his personal copy with the notes he had of the Memex paper. And he went on and took that idea and built the first kind of modern computers with a monitor and a keyboard and a mouse. And this really came out of the Memex.
37:00
Here's another picture of a system. This is citation analysis, which was, again, directly inspired by the Memex. And when Tim Berners-Lee came to invent the World Wide Web, he was, again, also very inspired by the Memex. So it was a weird little side project, and that's what I find so inspiring about this. So I think about side projects.
37:21
We're all busy, we have responsibilities. I think it's a big privilege to be able to work on side projects, and not all of us have that privilege. But on the other hand, does anybody follow this Twitter account, darkpatterns? So it just compiles all these tricky ways that developers spend time on tricking users into doing things they don't want. So I think about this, and I think there's a lot of surplus time available
37:41
for developers if you just phone it in a little bit when the boss asks you to make one of these. So in conclusion, here's how I use my own weird side project. So I'm coming to Portland. I've been here before. But I can quickly remind myself of all the times I've seen Portland things, and I can remind myself about the carpet. I can be reminded to take my own photo of the carpet.
38:02
I can see my friends who have been here and remind myself of where to go in the city. I know that I'm gonna run into people I've met before, so I can quickly pull up my notes and refresh my memory and have a very intelligent conversation with them. I can see where in Portland I've been to, which is nice, because I can sort of plan to be in a place I haven't been before. And there's a phrase in Portland
38:20
that's a bit of a cliche, but we all love it, I guess. Keep Portland weird. And I've been inspired by this, and I think we should keep our side projects weird as well. Yeah. So, thank you, that's my Memex. Thank you. Thank you.