We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Smartphone Symphony

00:00

Formal Metadata

Title
Smartphone Symphony
Title of Series
Number of Parts
27
Author
License
CC Attribution - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language
Producer
Production Year2018

Content Metadata

Subject Area
Genre
Abstract
If Beethoven was alive today, he’d probably use Ember. Using Ember, socket.io, the web audio API, and devices in the audience's pockets, we'll interactively explore the challenges of creating a real-time and audience-driven musical performance, different parts of which are played across a symphony of all phones in the audience.
SmartphoneLinker (computing)Decision theorySoftware frameworkMusical ensembleMultiplication signWeb applicationPerspective (visual)Greatest elementMobile appSoftware frameworkRight angleSmartphoneBitGUI widgetBoss CorporationCodeXMLComputer animation
Convex hullProduct (business)Touch typingTwitterResultant
Probability density functionHookingProduct (business)Address spaceNormed vector spaceComputer animation
Event horizon
Set (mathematics)Different (Kate Ryan album)Multiplication signBitEvent horizonBuildingSlide ruleLecture/Conference
Web applicationNetwork socketSlide ruleMessage passingReal-time operating systemMobile appServer (computing)LaptopComputer fontGoodness of fitPresentation of a groupNumberComputer animation
MereologyServer (computing)Interactive television2 (number)Point (geometry)BitCrash (computing)Multiplication signLecture/ConferenceComputer animation
Category of beingNetwork socketPrice indexElectric currentEvent horizonProcess (computing)Core dumpPoint (geometry)MereologyCondition numberGodServer (computing)Arithmetic progressionComputer animation
Slide ruleDefault (computer science)TouchscreenReading (process)Slide ruleData structurePoint (geometry)Source codeTouchscreenRootDifferent (Kate Ryan album)Presentation of a groupPhase transitionNetwork topologyComputer fileComputer animation
Slide ruleTouchscreenGame controllerComputer animationLecture/Conference
TouchscreenSlide ruleGradientDefault (computer science)GradientGraph coloringSmoothingGroup actionSlide ruleConcurrency (computer science)Task (computing)Connectivity (graph theory)Procedural programmingOnline helpComputer animation
Execution unitConnectivity (graph theory)Multiplication signTask (computing)Disk read-and-write headSlide ruleComputer animationLecture/Conference
Default (computer science)Connectivity (graph theory)CodeSlide ruleScaling (geometry)Web pageComputer animation
Pointer (computer programming)Server (computing)AbstractionVolume (thermodynamics)Demo (music)Multiplication signElectronic mailing listStructural loadInteractive televisionBuildingRight angleComputer animation
Computer animation
Multiplication signDrum memoryMessage passingModal logicComputer animationLecture/Conference
Multiplication signSoftwareBeat (acoustics)Different (Kate Ryan album)Message passingNatural number1 (number)InternetworkingComputer animationLecture/Conference
Different (Kate Ryan album)Multiplication signTraffic reportingSoftware testingQuicksortComputer virusComputer fontEntropie <Informationstheorie>RootGraph (mathematics)Computer animationLecture/Conference
Multiplication signServer (computing)Different (Kate Ryan album)MeasurementTotal S.A.Drum memory
Graph (mathematics)Multiplication signComputer animation
outputAndroid (robot)Computer animation
Different (Kate Ryan album)Android (robot)Multiplication signGoodness of fitMathematicsRevision controlLecture/Conference
Computer wormSheaf (mathematics)Multiplication sign2 (number)5 (number)Arithmetic meanDifferent (Kate Ryan album)Computer animation
Sheaf (mathematics)Right angleGradientSoftwareSelectivity (electronic)Computer animationLecture/Conference
Workstation <Musikinstrument>Musical ensembleDigitizingSoftwareWebsiteLogic synthesisDialectGame controllerTelecommunicationVideo gameComputer animation
Execution unitDecision theoryLie groupEvent horizonSquare numberRight angleRange (statistics)Arithmetic meanSound effectForm (programming)Presentation of a groupDifferent (Kate Ryan album)AreaPhase transitionNumberServer (computing)Musical ensembleMultiplication signLocal ringTouchscreen3 (number)Mobile appGraphics tabletGraph (mathematics)Parameter (computer programming)DigitizingCircleHeegaard splittingNP-hardSoftwareComputer animation
Linker (computing)Square numberAverageRSA (algorithm)Coma BerenicesComputer animationXML
Block (periodic table)Data typeComputer animation
Transcript: English(auto-generated)
Thanks for that private joke and the music.
So hey, I'm Gavin. I'm here to give a talk. It's called Smartphone Symphony. Before I begin, I'll just give you a little background to the talk, how it came about. I work at a company called Intercom. It's a really amazing company. You might recognize us from our messenger, which is this little widget in the bottom right-hand side of the corner,
which is embedded in many great web apps and mobile apps. I joined four years ago when we were pretty much tiny and was lucky enough to be the person to convince everyone to take a bet on Ember. I talked a little bit about this last year at my talk.
We've made so many mistakes in income. We make them all the time. But we, I think, catch most of them and fix them quickly. We also got a lot of things right, and taking a bet on Ember was one of the biggest things we got right. Andrei mentioned yesterday our code base has never been healthier, and it's four years old, which is great.
I've increasingly been spending a little more time reaching out and talking to different people who are considering choosing a framework. I think we've got a good perspective and a story to tell them. If you know anyone like this, feel free to send them my way. It could be colleagues or a boss or friends or whatever.
Anyway, our really great growth has resulted to us recently announcing that we're doubling our R&D team in the next 18 months. So we've begun hiring 350 people. A hundred or so will be engineers. Most of them won't have used Ember before, but hopefully a lot of them will be excited to be here next year.
Of course, if you're interested, please get in touch. Intercom is a great company, but we have aspirations to become a really great company, one of the great companies. I think you can see this aspiration expressed in things like the books that we produce, our brand design, or the amazing people that work there.
These are a collection of intercomics. Everyone gets one of these each year on their anniversary. It's personal. It contains references to the highlights from the year. These are my four intercomics, all Ember-themed. I think I'm the only one who has that.
I think you can also see this ambition in the events that we put on. So this picture is from one of our product-focused events in San Francisco last year, and in London, and in Dublin. And in November, we put on our first engineering event, again in Dublin, on the famous Ficker Street.
And we had a set of technical talks talking about how we approach building products in intercom. And this was really an amazing event. It was more like a Radiohead concert than a tech event. I wanted to come up with a talk that was slightly different.
I mean, I think it's amazing that we get to live in a time where it's possible to build an impactful career where the work doesn't really feel like work. And I wanted to build something fun and a little bit different. So this was the first slide of my talk, and it'll be a good intro to the rest of this talk.
My slides are an Ember app running on my laptop. I'm also running the same web app on my phone, which allows me to control the presentation. I'll soon ask you to open up the same web app, if you want, on your phones, so we'll all be connected together. When I...
Whoa, that's loud. So I'll be able to trigger sounds through real-time on your phones. And they'll send real-time messages through a simple Node server that's running in Amazon Portland. It's just a simple Socket.IO server.
So the talk that I gave, I gave a portion of this one time before, and this part went pretty well. This was a high point. I can't believe that actually worked. But then a few seconds later, it actually no longer worked.
So the server crashed. Let me just check something on the computer.
All right, so, I mean, the server crashed. I had no quick way of getting it running again. So I couldn't really complete the second part of my talk. Not a big deal. One of the themes of the night was risk-taking, and this was a little bit of a risk, and it was worth it. After I got home, I realized my mistake. I simply was missing a guard. It was some race condition, and the Node server blew up.
I haven't done much Node, so hopefully that's a lesson learned. We'll see. Okay, anyway, I fixed the bug, submitted the talk to EmberConf. Happily, I was accepted. My talk at this point was ten minutes long, so I needed some more things to talk about.
So, I don't know, we're seven minutes in, so. So I extracted parts of the talk into an add-on called ember-present. It's a work in progress. Presenting ember-present. Just like, you know, roots.js in your Ember app, there's a slide.js where you describe the structure of your presentation.
This file, there's three different roles, screen presenter and audience, and three slides. It's likely I'll use a DSL similar to router.js, or actually probably use router itself.
This is still an experimental phase. There's lots of to-dos and half-baked ideas in the source code. I've just about got it to the point where I could give this talk. There's a few people out there who know how close I actually came to not having a talk. So, okay, so there's a couple things you can do, like simple things.
You can change the background color, define the background color, and change it dynamically. Same with background images, and I'm just pressing buttons here. There's also video backgrounds. Okay. It's also easy to define nice gradients. I mean, CSS is great.
So, you might know this is nice, smooth color transitions there. Alex gave a really great talk last year about his even greater add-on, Ember Concurrency. And his slides were custom-built Ember app, which allowed him to use components,
which in turn used Ember Concurrency tasks to help visualize their many features. And Alex's talk was a real inspiration. To me, I came away wanting to be able to build talks like that and put Ember in my slides. So, here is, with permission, the same component running my slides,
and I'll just perform a few tasks here, which is nice. Cancel, clear, perform some more. I actually wanted to put in, like, I ran out of time. I wanted to put in, like, a mopey head, Photoshop spinning around, but I never got there. There's also components for putting in code, and this uses Edward Faulkner's Ember code snippet add-on.
There's handlebars and JavaScript. Slide scale, so just a simple CSS keeps them in a HD 16 to 9 ratio. There's also an optional real-time server, which allows you to do things like this.
You have a little pointer. So, you can see there's two people swimming down here. So, all right, live demo time. It's 2018. I think we've built enough abstractions upon abstractions that this thing is going to work. So, if you'd like to participate, could you take out your phone and load this URL?
You'll see up in the top right, there's a list of people connected already. Great. So, just put in your name, click next. If the name's taken, put in a different one. We will be triggering sounds in your phone, so please turn up the volume. If you have an iPhone, you may need to flip the little switch in the top left.
You can also press the buttons to test your sound, make sure it's working. I'm very happy that we've got this far.
How many do you think we might get to? I guess I'll wait until it stops going up. This is great. It's working. This must be like what Elon Musk feels all the time.
It's still going up?
Whoa, whoa, whoa, whoa, whoa. All right, let's go. Something weird's happening, so I'm going for it. Okay. Whoa, whoa, whoa, whoa. Okay. Okay, so this is a simple example of triggering real-time drums. Every time I press a button, it'll send real-time messages to your phones to play particular sounds.
Whoa. Okay. Notice that the drum sounds aren't playing all at the same time. We're all connected through, you know, different networks, but even if we're on the same ones, the nature of the internet, there's different latencies from message to message, and this is even clearer if I trigger a drum beat to play,
and whoa, this will sound pretty bad. Yeah. Okay. So we can visualize the differences in latency across different phones by running some latency tests. I'm unclear why these have already run, but I'm going to run them again.
So this is running latency tests on your phone. It's doing a lot of pings and pongs and measuring the amount of time that it takes. As reports come in, this is meant to be a sorted graph, by the way. It's not sorted. All right, but it looks like most people are under 50 milliseconds
as a few outliers. That kind of bodes well for the rest of the talk, even though there's weird stuff happening. One approach to dealing with latency ... I'm not pressing buttons. One approach to dealing with latency is to work out a shared understanding of what time it is across all devices, and if we can do that, we can schedule sounds to play at the same time.
So phones generally keep the same time, but there's still enough of a difference that it wouldn't work for playing drum sounds. What we can do, though, is ping the server and measure how long it takes, so let's say 20 milliseconds in total, and we can assume that the request out takes about the same time as the request back, so 10 each, and if we add that to the server time,
we now have a pretty decent understanding of what time it is on the server, and that's exactly what we did when we pinged all your phones. So this is a graph that is not working. Well, okay, maybe I can explain what this is meant to be. This is meant to show the drift in time across phones.
Possibly what's happening here is some phone is, like, two months behind or something like that, and it's just, you know, made everything tiny. So I don't know, maybe you're right there and you know who you are, so nice one, thanks.
Okay, so we'll use this offset to trigger some sounds. Let's see if it's a little better. Well, that's actually not bad, not bad. Let's try the drum beat. Okay, let's try just iOS and Android.
I mean, I love Android, but, you know, I don't think it's a real word, but if it was, I think this is a really good audio-ization of Android fragmentation.
Just like iOS, it's receiving messages, it's queuing up audio to play at the right time. It's just that there's different onboard latencies between different versions of Android and manufacturers, so it doesn't work too well. One of the reasons why the different lag times are so noticeable is that drum sounds
have a really sharp attack, meaning that sound all kind of happens at once. It shouldn't be as noticeable with sounds that are more gradual attack, like orchestral sounds. So I put together 12 different orchestral sounds from a section of Beethoven's Fifth, Symphony, which I'll now randomly assign
to different devices. All right, this is about 40 seconds long. Here we go.
I can't believe that actually worked. So now that we have the phones synced, let's add the venue speakers. So I'm going to randomly select people from the crowd, so if you would choose which side of the room you're in. If you don't want to play an EmberConf gig,
just click Nope. All right, this is a piece of software called Ableton Live. It's a digital audio workstation that people use to generally create electronic music. I've created a short track for this talk, which uses four kind of overlapping synthesizers, which you can see here.
It's like Prophet 3 and things like that. Hey, there's the guy who's ruined my graph. We got a laugh out of it anyway, so it was worth it.
The black area is on the right. This is also running an Ember app. I just have split screen. Whoa, whoa, whoa. Okay. It's like someone else has managed to log in as a presenter and is...
All right. It's moving on. So the area on the right is an Ember app as well, split screen. I'm also running a local node server, which is subscribing to certain events and just translates these into mini signals, which is a way of talking to digital music software
like Ableton. And I've also arranged it so these four synthesizers, as I said, can be controlled by four people. So let's pick someone, first of all, from... Okay, Dr. Less. So you should be able to move that around. Great.
So let me just solo pad one and press play. All right, so, yeah, you can just explore the four corners to see how the sound actually changes. That little green square will give you an idea of where, when I recorded it, where I put the parameters.
Music is expression, so go wherever you want, but if you ever feel lost, just put the circle beside the square. All right. Let's get a second person. Solo this, pad two. Okay, Dr. Less.
Okay. How can you move two like that with one person? Okay, I guess this is a different person, right? Is it a different person? Yeah. Okay, great. Okay, so let me solo you and press play here.
Okay, maybe move it over to the right, see what happens. Yeah, okay. Great, let's get number three. Okay, it's not Dr. Less. So VCFVBB.
Okay, I'm just gonna move on. Let's get number four going. Okay, profit V3. I might have to run one of like a thousand buttons here. Okay, we might have to go with three people.
Let's see, profit V3. Yeah, doesn't seem to be working. Okay, so let me make sure we're set up here right. I need to un-solo V2. Try this a little hard. We enable motion, turn off loop, go back to intro.
Okay, here we go. Here we go. Yes, it is. Thank you. Un-solo, thank you. Oh, it's Luke. Thanks, Luke. All right, let's try this again.
That's okay.
That's all I have. Thanks very much.