Template-first Data Visualizations
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 16 | |
| Author | ||
| License | CC Attribution 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/62174 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EmberConf 20227 / 16
5
6
8
12
13
14
15
00:00
AudiovisualisierungTemplate (C++)MathematicsVisualization (computer graphics)Different (Kate Ryan album)Software frameworkMultiplication signCodeFrequencyQuicksortWeb 2.0Selectivity (electronic)Semantics (computer science)Connectivity (graph theory)Revision controlComputing platformLattice (order)MathematicsCombinational logicMereologyTemplate (C++)Computer animationMeeting/Interview
02:12
Abelian categoryCartesian coordinate systemLine (geometry)Category of beingCartesian coordinate systemLine (geometry)QuicksortBit rateAverageCodeState observerLaptopAuthorization
02:40
Keyboard shortcutTorusLibrary (computing)Functional (mathematics)System callElement (mathematics)Source codeCartesian coordinate systemUtility softwareGreatest elementMultiplication signConnectivity (graph theory)CodePoint (geometry)RectangleScaling (geometry)Right angleGroup actionCompilation albumBit rateMeeting/Interview
03:59
Gamma functionEuclidean vectorConnectivity (graph theory)Keyboard shortcutRow (database)Element (mathematics)Category of beingComputer fileGroup actionQuicksortNumberMeeting/Interview
04:46
Abelian categoryStatisticsInverter (logic gate)Scaling (geometry)Category of beingCartesian coordinate systemArithmetic meanBit rateKritischer ExponentAverageQuicksortFunctional (mathematics)Discrete groupSet (mathematics)Term (mathematics)Primitive (album)CASE <Informatik>Level of measurementGreatest elementLinearizationMeeting/Interview
05:25
EmailFunction (mathematics)Euclidean vectorGEDCOMAverageClique-widthGroup actionArithmetic meanCache (computing)Cartesian coordinate systemPixelBound stateTime domainScale (map)Range (statistics)Abelian categoryMenu (computing)EmulationWorld Wide Web ConsortiumParameter (computer programming)Scaling (geometry)Functional (mathematics)Group actionElement (mathematics)Template (C++)Cartesian coordinate systemSummierbarkeitCategory of beingTrailSpacetimeDivision (mathematics)Different (Kate Ryan album)Object (grammar)Function (mathematics)Row (database)Clique-widthBound stateRange (statistics)PixelDomain nameBit rateState observerLoginVisualization (computer graphics)Video game consoleStatisticsConnectivity (graph theory)System callKeyboard shortcutSelectivity (electronic)Set (mathematics)Data loggerNumberChemical equationKritischer ExponentEntire functionMarginal distributionMaxima and minimaCASE <Informatik>Boundary value problemInsertion lossMeeting/Interview
08:35
Computer wormShape (magazine)File formatInterpolationUsabilityHierarchyQuad-BaumRandom numberInheritance (object-oriented programming)Point (geometry)Library (computing)BitException handlingVisualization (computer graphics)Different (Kate Ryan album)QuicksortPattern languageMobile appSubject indexingSoftware frameworkMorley's categoricity theoremOffice suiteInheritance (object-oriented programming)Optical disc driveDataflowNetwork topology1 (number)Cartesian coordinate systemRight angleComputer fileStapeldateiParsingControl flowMeeting/Interview
10:41
Attribute grammarElectric currentTime domainCode refactoringCloningEuclidean vectorGroup actionLine (geometry)Clique-widthMultiplication signCross-site scriptingCode refactoringRule of inferenceThumbnailPresentation of a groupTemplate (C++)QuicksortAttribute grammarElement (mathematics)BitMeeting/Interview
11:44
Abelian categoryPattern languageRight anglePoint (geometry)CodeOverlay-NetzBitNumberDistribution (mathematics)Line (geometry)Metric systemMeeting/Interview
12:28
Connectivity (graph theory)Code refactoringProcess (computing)Goodness of fitProduct (business)Computer animationMeeting/Interview
13:08
Abelian categoryStreaming mediaElectronic program guideMathematicsPoint (geometry)QuicksortCross-site scripting
13:39
Execution unitGamma functionAreaAbelian categoryCellular automatonMarkup languageElement (mathematics)Object (grammar)DatabaseCross-site scriptingMathematicsInferenceRule of inferenceCASE <Informatik>LogicCartesian coordinate systemProduct (business)Error messageAverageCellular automatonCategory of beingPersonal digital assistantTouchscreenTable (information)QuicksortProfil (magazine)Domain nameRectangleFormal languageSemantics (computer science)Level (video gaming)Schweizerische Physikalische GesellschaftLine (geometry)Meeting/Interview
16:03
Cartesian productConnectivity (graph theory)Pattern languageMultiplication signCartesian coordinate systemPlotterCode refactoringTerm (mathematics)SpacetimeCASE <Informatik>Cartesian productMeeting/Interview
16:57
Component-based software engineeringTransformation (genetics)Euclidean vectorHash functionConnectivity (graph theory)Inheritance (object-oriented programming)Category of beingLevel (video gaming)Parameter (computer programming)Cartesian coordinate systemScaling (geometry)Transformation (genetics)Object (grammar)Cartesian productMeeting/Interview
17:55
Type theoryCartesian productQuicksortValidity (statistics)DatabaseDifferent (Kate Ryan album)Standard deviationWeb applicationMeeting/Interview
18:34
Template (C++)Core dumpAudiovisualisierungSpacetimeMereologyComponent-based software engineeringAbstractionFehlererkennungLevel (video gaming)OpticsBlock (periodic table)FingerprintSheaf (mathematics)KnotLabour Party (Malta)Abelian categoryConnectivity (graph theory)Formal languageLibrary (computing)BuildingAverageTemplate (C++)Different (Kate Ryan album)Type theoryDatabaseQuicksortLimit (category theory)Multiplication signKey (cryptography)Web applicationCategory of beingRepresentation (politics)InformationProduct (business)Data managementWrapper (data mining)Point (geometry)Right angleArrow of timeSoftware frameworkLatent heat1 (number)Block (periodic table)Axiom of choiceMeeting/Interview
20:27
Pairwise comparisonProgrammer (hardware)Source codeParity (mathematics)Total S.A.Template (C++)Library (computing)Category of beingClique-widthType theoryPoint (geometry)Product (business)Data managementRight angleSummierbarkeitRow (database)Cartesian coordinate systemNumberGreatest elementComputer animation
21:22
Abelian categoryScale (map)Range (statistics)Gamma functionInformationConnectivity (graph theory)Library (computing)Type theoryProduct (business)Wrapper (data mining)Different (Kate Ryan album)Data managementDomain nameCartesian coordinate systemPosition operatorClique-widthParameter (computer programming)Latent heatSummierbarkeitOperator (mathematics)Cartesian productMeeting/Interview
22:35
Abelian categoryState observerGraph coloringData managementProduct (business)Goodness of fitNumberProcess (computing)Point (geometry)Software frameworkCartesian coordinate system
23:20
Numeral (linguistics)Template (C++)PermianPoint (geometry)Software repositoryCodeSingle-precision floating-point formatLevel (video gaming)Line (geometry)Presentation of a groupWeb 2.0Cross-site scriptingMeeting/Interview
23:51
Computing platformVariable (mathematics)State transition systemCross-site scriptingCodeLine (geometry)Web browserComputer hardwareDemo (music)Mathematical optimizationSinc functionMeeting/Interview
24:27
Hill differential equationMathematicsLine (geometry)Zoom lensCategory of beingCodeState observerExistenceInteractive televisionRun time (program lifecycle phase)QuicksortVariable (mathematics)Personal area networkLengthMultiplication signCartesian coordinate systemSocial classCross-site scripting
25:37
Gamma functionMassClique-widthFile formatExecution unitRule of inferenceMeta elementElement (mathematics)GEDCOMVideo trackingElement (mathematics)Variety (linguistics)NumberInterpolationParameter (computer programming)Object (grammar)ResultantMultiplication signFrame problemGroup actionCategory of beingSelectivity (electronic)Template (C++)BitLoop (music)Library (computing)Endliche ModelltheorieTask (computing)Game controllerCoordinate systemConcurrency (computer science)VolumenvisualisierungSequenceArray data structureGraph coloringString (computer science)Functional (mathematics)Thread (computing)TrailType theoryContext awarenessRight angleMobile appCross-site scriptingAttribute grammarReading (process)QuicksortMeeting/Interview
29:14
QuicksortPoint (geometry)Frame problemTask (computing)Parameter (computer programming)1 (number)Ferry CorstenTheoryConcurrency (computer science)Meeting/Interview
29:54
CodeLogicSoftware framework2 (number)Multiplication signTemplate (C++)Observational studySoftware frameworkCodeCartesian coordinate systemField (computer science)DatabaseCodecIntegrated development environmentSoftware maintenanceLevel (video gaming)QuicksortType theoryComputer animation
30:40
Forced inductionUsabilitySystem programmingSoftware testingAudiovisualisierungData structureVisual systemSemantics (computer science)EmpennageIntegrated development environmentProduct (business)Common Language InfrastructureAndroid (robot)Computing platformData dictionaryPhysical systemFormal languageRule of inferenceNormal (geometry)Module (mathematics)Function (mathematics)Server (computing)Variety (linguistics)CodeContent (media)FeedbackCommunications protocolStability theoryHuman migrationConstructor (object-oriented programming)Fiber (mathematics)Game theoryPrototypeCloningGrass (card game)BuildingWeb pageObject (grammar)Component-based software engineeringEvent horizonVisualization (computer graphics)Right angleType theoryConnectivity (graph theory)Markup languageQuicksortToken ringAudiovisualisierungComputer architectureCross-site scriptingGraph coloringFreewareDemoscenePhysical systemSoftware frameworkPerspective (visual)Endliche ModelltheorieCombinational logicParameter (computer programming)Internet service providerResultantFiber (mathematics)System callCuboidTemplate (C++)Computer programmingMeeting/Interview
32:21
Supersonic speedCodeRange (statistics)Classical physicsElectric dipole momentCore dumpFreewareNumberTerm (mathematics)FreezingConnectivity (graph theory)Field (computer science)MultiplicationModal logicMultiplication signMeeting/Interview
32:53
Meeting/InterviewComputer animation
Transcript: English(auto-generated)
00:15
Hi everyone, I'm Michael Lang. I'm an engineer at HashiCorp, and today I'm excited to talk about
00:20
template-first data visualization. We're going to be looking at two different tools, EmberJS, of course, that's the conference we're at, and also D3.js, which remains the sort of state-of-the-art way to visualize data on the web. But first I want to go back in time a little bit and recognize that Ember and D3 are both over 10 years old. Both of them released initially in 2011. And of course the web has
00:44
changed a lot over this period of time. The way we write, the way we package, the way we deliver our JavaScript code has all evolved. And also what users expect from the web has evolved with it. Here we've plotted all of the major versions of D3 and Ember. When it comes to D3, the most major milestone was D3.4, where D3 went modular, sort of recognizing that JavaScript
01:06
frameworks are going to be controlling your code, and then maybe people don't want to use all of D3's selection join sort of semantics, and would instead rather just use pieces of D3 rather than the whole thing at once. And for Ember, of course, Ember's changed a lot over
01:22
this period of time, but as it relates to this talk, I think it's interesting to look at how templating has changed. HTML bars, Glimmer, Glimmer 2, and Glimmer components, all milestones as it comes to not only templates, but also as we progress towards native JavaScript using parts of JavaScript that didn't even exist in 2011.
01:41
And there's a combination of both new features that are being added, and also huge performance improvements that let us realize ideas we had back in the day, creating more sophisticated compositions. And I say this all because you may be wondering why now. This isn't the first data visualization talk. It's not even the first immigrant data visualization talk, but as long as our tools change, and as long as the web platform continues to change as well,
02:04
these talks will always be relevant. It's always useful to sort of look back and see if the best practices have changed and what we can do today that's different than yesterday. So let's just start with the bar chart. This is the quintessential D3 bar chart. I actually just took the code straight from the observable notebook that the D3 authors
02:22
published, and then I replaced the data with data from Ember observer. So here are the average rating of popular add-ons by category, sort of descending by average rating. And if we look at this, we see bars. Of course, it's a bar chart. We also see an x-axis, a y-axis, grid lines, and this label of average rating in the top corner. And if we look at the code,
02:42
first off, it's all JavaScript. Start off, we transform data, we create these utilities, the scale and axis functions. And then we actually just create our SVG element with JavaScript right here. const SVG equals D3 dot create SVG. That's creating an SVG. If you've been around for a while, you may have noticed that this API is inspired heavily
03:03
by jQuery. In fact, at one point D3 was even using the Sizzle library that jQuery was also built on and authored. And then, notably, at the bottom here, we're binding our data by calling dot data, and then we are joining that data to DOM using the join call.
03:20
And here we're doing an imperative data binding to recco elements. And once that runs, we get this SVG element in the DOM. Immediately, we can notice how different this looks from the source code. But we can also see that SVGs act to look like HTML if we haven't spent a lot of time looking at them. And if we squint, we can sort of see the components. Here we're creating the SVG, and then we also have our y-axis. We know it's
03:43
the y-axis because they're near the bottom. We have that text element that says average rating. After that, we have a G, which stands for group. And within our group, we have all these rectangles. And, of course, the rectangles are going to be the bars. Lastly, we have our other axis down here. So let's try this again with Ember. We know that this ends up looking like an SVG,
04:03
so can we start with it maybe looking like an SVG? So here we go. It's a Glimmer component. Glimmer already has fantastic support for SVG, so there are no gotchas here. We just create an SVG element in our HBS file, and we combine stuff to it. Inside of this, let's just start with the bars. Of course, we're going to create a group element, and then we're going to iterate
04:23
over some sort of array. And the way we do that in Ember is using each. And then inside of that, we're going to create rec elements and then bind some data to them. Lastly, we need these axes. Axes are a little more complicated, and D3 already has a solution. So let's just put this in here and hope that we can continue to use D3 for that. And now the only thing
04:41
that remains is we have to define these properties somewhere. So let's do it. Let's try joining D3 and Ember. We're going to look at scales, axes, and statistics here. Let's break down our bar chart in terms of D3 primitives. First, we have an axis left. This axis left
05:00
is built on top of a scale linear of 0 to 10. It's not logarithmic or anything like that. Then we have an axis bottom down here for our x-axis, and it's built on a scale ordinal. This is a discrete set of data based on category name. And then this is an average rating of popular add-ons. So we need to construct some sort of average function. In this case,
05:20
the mean before we apply the scale functions to our bars. I'd be remiss to not show you the data of a data visualization. So the keen eye will recognize this as the console log output of an Ember data record or a record array. And here we have an array of categories.
05:40
Each category has an array of add-ons. And then unseen inside this object is a score property on each add-on. So let's start off by just creating ourselves a Glimmer component, a bunch of track properties, et cetera, and some getters. Notably in here, inside of our data getter, we are already using D3. We're going to be using D3.me, a function of the
06:02
D3 array package, to be able to construct that average score from our array of elements. Summation division isn't so hard, but starting in D3, might as well use it. Next, we're going to use the D3 scale package. Just to explain this really quickly,
06:20
basically what we're doing is we're taking our pixel bounds here as range, which is just the difference between the height and our margins. And then we're going to map that to data bounds. In this case, it's a fixed bounder of 0 and 10, which is the minimum and maximum rating that could be achieved using Ember Observer. And data bounds here can also be ordinal.
06:42
So as mentioned, the scale, the x-axis is ordinal. How do you map pixels to ordinal data? You provide the entire set of values to the domain function, as we can see here where we're saying return D3.scaleband.domain. And then lastly, we have our axes, which are using that D3 axis package, and they're going to create SVG elements for us. Let's look at the data first.
07:06
So as mentioned, we have that the x, y, height, and width properties are applied to our bars in our template. And it turns out we can just use a getter here and then just call our D3 scale functions. Since they're just functions that map values to pixels, that's data that
07:22
operates on data that we can then use in a template, which is perfect because that's how all of our data operates. Everything in Ember these days is around the concept of derived data. The last thing we need to do, though, is get those axes in. And the issue is that they don't return data, they return actual SVG nodes. So somehow we have to take those SVG
07:40
nodes and get them into our DOM. And we could use a modifier for this. So by having a data insert modifier and a calling or mount elements action, we have a handle on the SVG element itself that we can then use with D3 to hopefully get those nodes into our DOM. So here we can see that we're creating an element const $l equals d3.select. Wrapping
08:01
the selection around the element gives us the D3 API to then select child elements and then construct the axis and then bind it into our existing SVG element. So just to recap really quickly, we use the D3 array package, which is full of statistics helpers including d3.meme. We then use the D3 scale package,
08:23
which gives us mapper functions to translate from data space to pixel space. And then we use D3 axis, which is a DOM emitting function that creates the marks for axis prepositioned according to scale. Then we're able to mount that into our component. At this point, I think we should just dissect D3 a little bit. As I mentioned at the top, D3 is now modular.
08:41
It's a library of libraries, and there are truly no exceptions to this. This is what the index.js file looks like for D3. All it's doing is exporting the exports from its sub-packages. So let's break this down. I like to think about how to categorize D3 packages using the definition of D3 to begin with. D3 stands for data-driven
09:03
documents. So we have packages for fetching and parking data, for transforming data, and then lastly for drawing things to the DOM via HTML, SVG, or Canvas. So if we take that categorization and then organize all of the exports from that index.js file, we end up with a situation
09:21
like this. A couple of data packages, a whole bunch of these driven packages, and then a handful of document packages. Just to further visualize this, this is kind of the flow that your data is going to use. You're going to use that data package to hit an API or maybe just that static file, and then you're going to use driven a package here to transform what may be flat data into this
09:44
actual tree construct with children and parent nodes. And then lastly, the document packages are going to be the thing that takes this transformed data and turns it into a visualization. And here, if we're thinking about this with our Ember glasses on, I think we can all agree
10:01
that we don't need these data packages. Odds are you have some sort of data fetching pattern in your app. Maybe it's not Ember data, but I'm pretty confident it's not going to be D3. Over here with these dominating ones, it's a little weird. Ideally, we don't do this, right? But there's also a lot of value in these, so we don't necessarily want to throw them away. We already used the access one, and it worked just fine. So let's just sort of keep that under
10:24
a hat and think about it. But really, the magic is going to be in these driven packages. This whole collection of transforming data into different data works perfect not only for D3, but also for any sort of JavaScript framework. And of course, this is by design from the office of D3. I'm going to look at that chart one more time, because we haven't talked about CSS yet.
10:45
And CSS isn't just for HTML. It also applies to SVG. So of course, we have these inline styles. I'm sure you saw those already. And that's just because we copy pasted straight from our JavaScript-only solution into our templates. As we go through this refactoring, we want to
11:00
move things into CSS, which not only includes these inline styles, but also presentation attributes within our SVG nodes. Just as a general rule of thumb, if something is presentation in SVG, there's likely to be an equivalent in CSS. And we also had styling going on in JavaScript. Of course, originally we had everything happening in JavaScript. But now we have this issue
11:21
where we have all sorts of little nuance-y things happening inside of our mounted elements, and it's a bit of a shame. But there are tricks we can do in CSS to sort of eliminate all of this twiddling that we're doing here and instead doing stuff in CSS. So voila, just like that, we can move all that stuff into CSS. And this ends up being
11:42
kind of familiar. At the end of the day, ta-da, we have a chart. Looks just like our other chart, which was the whole point, right? We didn't want to create a new chart. We wanted to create the existing chart with new patterns. And let's look at how the code ended up shaking out. Just D3 on the left, D3 plus Ember on the right. Immediately you'll note that the Ember
12:01
solution has more lines of code. Personally, I don't think lines of code is the end-all, be-all metric of whether or not code is good. I'm sure you can agree with the bear. So let's just do an overlay here to do a bit of a vibe check. In blue we have markup, in red we have JavaScript, and in yellow we have styles. So we've created a much clearer distribution of code across these different disciplines in our newer solution, even if it does
12:24
end up with more code. And I can hear you already asking why, though. If the D3 chart already worked just fine and we were able to sort of shove it into an Ember component, why go through this refactoring process? It just seemed okay. And the beauty of components is once it's in there, you only have to think about how it's implemented. It's
12:43
quite well encapsulated. And it's a good question. There's this old proverb, if you want to go fast, do whatever you want. If you want to go far, you separate your concerns. And now keep in mind, sometimes going fast is the right thing to do. I do not want to stop you from going fast if that's the right thing to do. But let's just talk about the value that
13:04
comes from wanting to go far and separating those concerns. So in this scenario, imagine a broad designer walks by and they see this chart. And they're thinking, that's a nice chart, but the text is so small. Can the people watching the stream even see it? So they bump up the text size. Simple CSS. And then they're thinking, that orange looks really bright.
13:25
Is that actually using the style guide orange? And it's not. So they make it match sort of the theme orange of the conference. And the point here isn't that CSS is easier than JavaScript. These changes are one liners in JavaScript more than CSS. But the point is, CSS is familiar,
13:41
which is not quite the same as easy. If you're a product designer, you're much more likely to be familiar with CSS than you are with JavaScript. For me, making a one line change in JavaScript is easy. If somebody who isn't familiar with JavaScript tries it, the easy thing to do is end up with a syntax error. Let's just walk through another example. In this case, we have an accessibility specialist
14:01
walk in. And they see SVG. SVG looks like HTML. This is all markup. It's familiar. Now, making a database accessible is highly situational, because charts can be read in so many different ways and have so many different intended purposes. So there are going to be many criteria to consider. What I like to do, especially for bar charts, is ask myself,
14:20
what would this look like if it was a table of data instead? And taking that logic, what we end up with is first giving a role to the SVG. Mind you, the SVGs aren't normally graphics. So we want to make sure that any sort of assistive technology will know that this isn't just a graphic. It is meant to be interpreted as well. And then we give the SVG a title
14:42
and a label, which is going to be read out. We hide our axes. This might be weird because there is text in here. But keep in mind that the ticks of an axis are not helpful at all unless you're actually using them in a visual manner. Having a screen reader announced 10.9.8.7.6 is not useful when you can't actually sort of visually map where the six is to the height
15:03
of a bar. And then we want to give our recs a cell. This minimizes the profiles announcements rather than saying yada yada and then rectangle is going to say just yada yada. In this case, it's going to say the category name as well as formatted average score.
15:20
Keep in mind here that both the x and the y values are important. Once again, without having the axes to be able to draw visual inference, we need to make sure this is clear in the label. And there's a lot that goes into this, as I mentioned, but the one hard and fast rule when thinking about data biz and accessibility is that doing nothing and hoping for the best
15:41
is never going to cut it. Keep in mind SPG is a graphics language. Semantic markup isn't going to be nearly as reliable in this domain. It wasn't really a design goal. HTML is going to be much better for that. In fact, there's an element in SPG called foreign object that lets you embed HTML within your SPG to be able to get better interactive and accessible markup.
16:04
Okay, with that out of the way, let's try extending. Components compose, of course, so let's explore compositional patterns with charts. All right, so here's our chart one more time. It plots data in xy space, so maybe we could think of this in terms of a general
16:22
purpose Cartesian canvas, and then one of the marks that can go on a Cartesian canvas happens to be bars. So then we also have our axes. Before we implemented this as axis left and axis bottom, but nice thing about components is that we can start broad and then over time
16:40
refactor and do something that's more specific as new use cases arise. In fact, I think in our first go at this, we only need the axis components. Let's just try thinking of it as just a Cartesian canvas and see if we can get away with having the axes derived from the data itself. Okay, so I pulled this component out of the oven which takes this
17:01
contextual approach. The beauty here is that we're going to compute the common scales and transformations in this parent level component, and then we're going to yield the bar component which is partially applied using contextual components. So if you've never used contextual components or if you don't mind the refresher here, what's happening is we're yielding just
17:21
any component would, but the thing that we're yielding is an object, and that object has a property bars. The value of bars is a component, and that component already has the data x-scale and y-scale arguments pre-applied, which means when we go to call this component, it just looks like this. The Cartesian chart is where we set our data and our xprop and all
17:41
those data related properties, and then once that's set, all we have to do is say c.bars to say we want that style of mark in our chart, at least constructing a chart much less involved. And the neat thing here is that you'll find the Cartesian canvas is the base of so many chart
18:00
types. This is just a smattering of 15 here, but there are so many more, and it's exciting that if you wanted to create a Gantt chart or have one of these population pyramids, these types, you already have the foundation within the Cartesian canvas. And I can hear you again. Once more, you're probably thinking, why though? Is there actually value in
18:24
doing all of this, or is this just sort of my fun experiment? And it's valid. Once again, database validation isn't exactly new, so there's all sorts of different approaches. I really like this article in Nightingale called Navigating the World of Web-Based Data Viz, and in here it's sort of plotted all the different types of chart libraries that exist.
18:43
And what I'm proposing here is down here in the framework-specific low-level building block that happens to use composed contextual components as our language tool of choice. And you might be thinking, but why not up here? Why wouldn't I want to use one of these chart template libraries? It even says it's less effort.
19:03
And you're totally right. It's less effort. And once more, if you want to go fast, do whatever you want, and sometimes going fast is the right way. If you're working at a startup, no one's going to remember your failed startup because it had beautiful chart components, right? Prioritize appropriately. But the reason why you may not want to do that is hiding under that blue arrow, and that's because chart templates themselves
19:23
are less expressive. And chart templates are great all the way up to the point where you really need them to be great. And once a chart component can't do the small thing that you need it to do, it can feel like you've just locked your keys in your car. And if you
19:41
solved that you can't, and you're not about to punch through the glass, that would be uncomfortable as well. So let's just walk through how quickly this can happen with a chart template library without throwing any shade on chart template libraries. They do what they do, and they know that there are limitations. So in this example, we're now thinking about
20:01
how a chart can change over time. So imagine a product manager comes into the room and thinks, oh, I've been thinking about this chart. And one of the shortcomings of this chart, if you look at it, is that there's no sense of how many add-ons are in a category. Sure, data has high marks here, but maybe there are only a few data add-ons in something
20:21
like library wrappers, or a lot of them, and that pulls the average down. So is this really a fair representation of that information? And they happen to see this chart from the economist. At this point, you're not sure if this is a real request, or if just a humble brag that your product manager reads the economist. Whatever. They talk through this, and they show how nice it is that this is doubly encoded. So not only do we see
20:44
a y value, but we also see a width on these bars, and it seems perfect for our data. What if the width of a bar could be based on how many add-ons are in a category? And we could think through this just fine, right? So our ordinal x-axis on the bottom turns into a
21:00
linear one, and then our width is based on the number of add-ons, and then our placement, our x value, is going to be the cumulative sum of all add-ons before it in the record array. So none of that is particularly hard, but if we're using a chart template library, and it doesn't know how to create this type of chart called a Marimeko chart, well then you're sort of stuck. But we're not, so we can create it. Voila. Just like that,
21:25
we have now created a component that takes a different type of x-axis, and then can encode it. And we can see that there are indeed a lot more library wrappers than there are data add-ons. But, you know, so that shows that maybe our product manager was onto something, and we're
21:42
now presenting more information to this chart consumer. And it didn't actually take that much work to do. We're still using that Cartesian chart as a base. It has a Marimeko-specific argument now with this x size prop, so we can compute that x width or the width of bars. And then we are now using this Marimeko-yielded component.
22:05
And what we have to change in Cartesian chart, we're getting more utility out of d3. So you can see in here that we are using d3.sum inside of that x domain sum, which is just going to aggregate data for us. Not a particularly difficult operation, but it's nice that d3 does it.
22:22
And then we're also using d3.cumulativeSum, which is going to give us the value of data aggregating from one position to the next position, which is precisely what we needed to get the x coordinate of bars. But, I mean, at the end of the day, we created the chart, sure. But is it good? I mean, if I'm looking at this, the direct labeling is nice, but it's a little
22:44
clunky, especially if you look at the x-axis, some of those numbers are kind of smushed. Also, the single color just kind of makes it seem bland, a little bit monolithic. But that's kind of also not really the point. What we're trying to do is set up a framework for creating a good chart. Creating a good chart is actually an interdisciplinary activity.
23:03
As we already watched, there was engineering involved, design involved, management or product management involved, as well as accessibility. So the engineering goal here isn't to create a good chart, but it's to support the iterative process that leads the whole team down to something good. At this point, I want to mention that everything so far has been
23:24
high level, even though there has been a lot of code. I don't expect you to pour over every line while I'm presenting. We're just going over concepts. There's a repo already online where you pour over every single line of code. If you have any questions, please reach out in Discord. Okay, now let's talk about animations. First off, in the year 2020-2022, the golden rule is
23:47
to just use CSS. If you're animating something on the web, just use CSS. There are lots of benefits here. First, with just a couple lines of code, you can have a really smooth animation. Also, the transition system within CSS is super durable, resilient to cancellation and updating
24:02
and oftentimes it's going to be hardware accelerated. It depends on the prop, but since CSS is declarative and is also owned by the browser itself, you can expect much more optimized code than something that would happen in user land. It's also using the platform, which means less trials could we have to write or maintain or download, and CSS variables make this more powerful than ever. I have a demo to show about how CSS
24:24
variables can combine with CSS to make something great. So here what we have is a simple line and a common animation with line charts is to sort of animate the line coming into existence or out of existence. If we quickly look at this code, we have a class line in it, and then we have the style dash dash length that's setting our property,
24:43
and then we have this D attribute, which looks super complicated and it kind of is. There's a DSL with an SVG for basically drawing lines, but the nice thing here is that there's also a stroke dash array property which operates on simple data, which means I can just click this and it's going to animate. And the missing piece of the length, that can be computed at runtime
25:04
when we generate a chart and then apply it as a CSS variable so we can still use that in our animation, which is super nice. There are other ways that line charts are animated, so let's look at this. This is unobservable once more. Observable is a great resource for learning D3. Lots of lovely interactive examples. So here we have a chart and we can filter
25:25
into a year, and watch what the x-axis does. Things are coming into existence, they're panning left and right, we're zooming in, we're zooming out, and this is all stuff that you're going to have a hard time trying to do with CSS. Because the issue here is that we now need to coordinate animations and we need to manage elements coming into and leaving the DOM.
25:47
So let's look at what we're doing here with D3. How does D3 manage to achieve this in a way that CSS cannot? D3 has a library called transitions, which mimic the D3 selection API. So for any selection, we can call dot transition, give it a duration, and then that's going to
26:01
automatically start animating the data that's bound to it. We can also coordinate across selections. So you can see down here, we're creating this transition as t, and then we're applying that transition to other selections, gx and path, to make sure this is all coordinated. And D3 will also smoothly interpolate various types of data. So that D attribute that we saw that was complex, not a problem for D3. But if we're using this,
26:26
who's in charge? It starts to feel a little funny because if you want to use D3 transition, that means you need to use D3 selections. And if you're going to be using D3 selections, that now means you're constructing DOM in JavaScript rather than in the template where we want to be. So let's just examine this a little further. If we remember, D3 is a library
26:44
of libraries. So D3 transition itself is in this document category, but it's powered by another D3 library called interpolate, which just operates on data. And D3 interpolate here, it says right in the README that it provides a variety of interpolation methods for blending between two values, and those values may be numbers, colors, strings, arrays, or even deeply
27:04
nested objects, which is fantastic. So that's our first requirement, smoothly interpolating data. The next one is coordination. But the thing is, coordination is actually just a concurrency problem. So how about emergent concurrency? That's kind of our go-to solution for coordinating work that's asynchronous. So with that, we can control data over time.
27:23
Ember is already handling read renders, so that's kind of already solved for us. And then emergent concurrency can orchestrate sequences of behaviors. I don't know, let's just try it out, right? So this is what a task would look like. We want to run this task until timer runs out. So we have a duration of 500. We use the high-performance timing API, performance on now, to figure out how long this has been opened. From there, we can compute
27:45
how long until we hit our duration inside this while loop. If you haven't seen these tasks before, you might be a little weirded out, because this is a while loop, and whiles are normally synchronous. But since this is inside of a generator and we're using yield, we can actually use the spend execution, which is great, by waiting for this request animation
28:04
frame. A little bit of a gotcha, though. Requesting animation frame actually uses the style of asynchronous modeling. Fortunately, it's pretty straightforward now to quote-unquote cast a callback style of asynchronous behavior into a promise-based one. So with this one-liner,
28:21
we now have a promise that resolves when we get a new animation frame, and then we can yield to this promise and have the results we want. So that takes care of the timing. Now we need to interpolate. So we just thread in d3 interpolate. Assuming that our data doesn't have references, then we can just take any object and interpolate between one to the other.
28:40
The interpolator is a function that takes a numeric argument between zero and one, which is basically the percentage through the interpolation that you want to be at, which is going to be now over the duration. Pretty straightforward. And then the last gotcha here is that even though we're setting a track property to make sure that we're updating the data and that's going to cause a re-render, it's not going to update our axes, because remember
29:04
that those are controlled by d3, which is not track property aware. But that's fine, we can just call this on app elements again, and it ends up being not an issue. So let's look at this. All right, here is that Marty Mako chart. Now we have a sort button, and when I click it, it's going to smoothly animate between two things, using request
29:23
animation frame and ever concurrent to task. Of course it's modeled, but that's not the point. The point here is that we can create our own animation parameters from existing Ember add-ons. So if you're going to go down this road, you're pretty quickly going to run into the questions of what happens when new data points get added, and also what happens when ones are
29:40
removed. There are concepts inside of d3 called entries and exits to manage this exact behavior. So in theory you could add that to some sort of Ember concurrency like API, but I'll warn you now that if you go down that road, you're just going to end up at Ember animated, so use Ember animated. And I can't believe that Ember animated only got, what, four seconds of time
30:04
in this talk, but the fact of the matter is DataVis is its own deep, rich field of study, and the intersection of framework thinking and DataVis thinking is actually pretty small. The bulk of DataVis right now comes from BI tools, where it's all sort of about creating those chart templates, and then storytelling and journalism, where we have really rich,
30:23
unique visualizations, but in an environment where speed is of the essence, getting a story out on time, and then once that story is created, it's sort of locked in. So there isn't, at the chart level at least, that type of sustainability and maintenance that needs to go into application code. So thinking about this intersection and looking forward, there's a lot
30:44
of stuff that I think is really exciting. Here's chartability. Chartability is sort of right on the edge research about accessibility and how that intersects with data visualization. So what's at the intersection of accessibility, data visualization, and frameworks? Can we take
31:01
these best practices and then embed them in our add-ons to make it so people get this type of behavior for free? Or what about design systems? Does your design system have design tokens? And if it has tokens, are those tokens available only in CSS, or can you also get them in charts? And then also does your design token handle colors as far as color ramps go, how data
31:25
visually exists? Think about color. And then also sort of from the architecture framework perspective, we mentioned earlier about components and contextual components. These can be kind of tricky because they hide a lot of what's going on beneath the scenes when we're partially applying arguments, but tools like Glint that's going to provide
31:42
IntelliSense for components, I'm very excited by this. And then thinking about performance. React3Fiber is fascinating because it takes the React template programming model, but it actually renders to a WebGL canvas where there is no DOM. And oftentimes data visualization, we end up
32:00
with performance issues where we would like to be able to either use a canvas or use a combination of a canvas and an SVG and HTML to be able to construct something. So is it possible for us to be able to create components that instead of emitting DOM just call canvas methods? And once we sort of open the box of markup that doesn't actually result in DOM, can we think about
32:23
compositions of audio as compositions of components and in that way create data sonifications? I think this could be really interesting. And it's also one of those things that intersects with accessibility to be able to make sure that we're expressing the field of data and multiple modalities. I know this has been a lot, maybe even too much for 30 minutes.
32:47
But at the same time, there is also just going to be so much more and there's so much more to build. And this is the perfect time. It's just the beginning. And now we get to build the future. Thank you.