We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback
00:00

Formal Metadata

Title
WebMIDI
Subtitle
The garlic bread of the music industry
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
The connection between Web and MIDI seems as likely as the joining of Garlic with Bread! Yet, we now have the power to create music from the web browser! Either by generating MIDI files for later manipulation, or as live instruments, WebMIDI provides us with the power to build some amazing online music applications. In this short form talk, Steven will cover the internal details of the MIDI protocol, and its related file format, to show how music is represented by computers. This details both the recording and playback of music within the browser, and external application. It comes complete with live demos, synthesized sounds, and occasional music jokes!
33
35
Thumbnail
23:38
52
Thumbnail
30:38
53
Thumbnail
16:18
65
71
Thumbnail
14:24
72
Thumbnail
18:02
75
Thumbnail
19:35
101
Thumbnail
12:59
106
123
Thumbnail
25:58
146
Thumbnail
47:36
157
Thumbnail
51:32
166
172
Thumbnail
22:49
182
Thumbnail
25:44
186
Thumbnail
40:18
190
195
225
Thumbnail
23:41
273
281
284
Thumbnail
09:08
285
289
Thumbnail
26:03
290
297
Thumbnail
19:29
328
Thumbnail
24:11
379
Thumbnail
20:10
385
Thumbnail
28:37
393
Thumbnail
09:10
430
438
Slide ruleCommunications protocolMIDIGoodness of fitMereologySlide ruleBitLevel (video gaming)HypermediaMIDIWeb 2.0Communications protocolKeyboard shortcutComputer animation
Slide ruleSoftware developerGame theoryEmulatorSlide ruleTheoremFigurate numberTouch typingProcess (computing)Software developerSign (mathematics)Term (mathematics)MIDIWebsiteTheoryComputer animation
MIDIFile formatComputer hardwareCommunications protocolSample (statistics)Message passingVolumeFunction (mathematics)Junction (traffic)Web browseroutputMIDIComputer hardwareFile formatWeb 2.0Standard deviationCommunications protocolCodeSequenceData storage deviceMessage passingNeuroinformatikLogic synthesisTrailForm (programming)Volume (thermodynamics)Latent heatWeb browserQuicksortOcean currentBitMereologyDecision theoryHexagonNumberRevision controlMultiplication signSerial portSingle-precision floating-point formatComputer programmingWordPoint (geometry)Video game consoleNavigationVideo gameArithmetic meanMultiplicationVirtual machineMatching (graph theory)Public key certificateComputer clusterMathematicsComputer animation
MIDIHypermediaMultiplication signJava appletScripting languageOrder (biology)Video gameWindowMessage passingWeb browserMIDIDemo (music)Particle systemInformationBit rateVolume (thermodynamics)Web 2.0Context awarenessCommunications protocolGreatest elementBitComputer animation
Message passingMIDIParameter (computer programming)MathematicsPressureRange (statistics)Meta elementEvent horizonType theoryChord (peer-to-peer)Analytic continuationBinary codePoint (geometry)MereologyOperator (mathematics)Electronic mailing listMIDILatent heatNumberRevision controlLibrary (computing)Communications protocolMessage passingLogicComputer programmingParameter (computer programming)BitMultiplication signDatabaseSystem callFundamental theorem of algebraSingle-precision floating-point formatPrice indexPressurePhysical systemString (computer science)Set (mathematics)Exclusive orStreaming mediaVolume (thermodynamics)Computer fileKeyboard shortcutMathematicsLevel (video gaming)Patch (Unix)Web 2.0Presentation of a groupChord (peer-to-peer)CodeComputer clusterMeta element1 (number)Event horizonAreaTouch typingComputer animation
Form (programming)Condition numberWaveMultiplication signHarmonic analysisWeb 2.0BitWeb browserKey (cryptography)VideoconferencingForm (programming)Message passingFront and back endsComputer animation
WaveformWeb 2.0WaveMultiplication signHarmonic analysisHookingGame theoryCodeSensitivity analysisGame controllerLoop (music)WindowThomas BayesNumberWeb pageWeb browserMessage passingMedianOpen setComplete metric spaceSampling (statistics)FamilyVideoconferencingEnumerated typeVirtual machineWordMobile appKeyboard shortcutMIDIRoutingGraphics tabletVariable (mathematics)PasswordComputer animation
MIDIPointer (computer programming)Form (programming)Drum memoryExecution unitDigital image correlationMagneto-optical driveFirst-order logicWindowVideoconferencingRootSquare numberReal-time operating systemWeb 2.0Point (geometry)CASE <Informatik>Multiplication signLevel (video gaming)NeuroinformatikMetropolitan area networkFamilyConnectivity (graph theory)QuicksortSoftware bugMIDIComputer animation
MIDIPrice indexTwin primeComputer multitaskingLibrary (computing)Design by contractGraphics tabletMultiplication signHypermediaBit rateMIDICommunications protocolWeightGame controllerMaxima and minimaExecution unitQuicksortKeyboard shortcutComputer animation
AbstractionLink (knot theory)outputFunction (mathematics)Expandierender GraphQuicksortPrice indexNumberNoise (electronics)Virtual machineWeb browserLibrary (computing)Web 2.0Computer fileFunction (mathematics)Physical systemSlide ruleExclusive oroutputMIDIValidity (statistics)Keyboard shortcutLink (knot theory)AbstractionComputer animation
WritingGraphical user interfaceProcess (computing)Library (computing)Point (geometry)ArmHypermediaMultiplication signStreaming mediaCASE <Informatik>Cartesian coordinate systemCommunications protocolVolume (thermodynamics)Musical ensembleEvent horizonExecution unitProjective planeAbstractionGame theoryText editorFunctional (mathematics)Series (mathematics)Physical systemBeat (acoustics)MereologyWeb 2.0Message passingSocial classLogic synthesisForcing (mathematics)Translation (relic)Different (Kate Ryan album)MIDIFault-tolerant systemBand matrixBit12 (number)Head-mounted displayEmulatorStability theoryComputer animation
Open sourcePoint cloudFacebook
Transcript: English(auto-generated)
Good morning everyone as that part of the slide says my name is Steve and as that part of the slide says I'm a little bit of a geek if we've all turned up to the right room I want to be talking about web MIDI for the next 20 minutes or so We're gonna go through who I am what I've done to be on the stage
What is MIDI in itself so we can talk about what is web MIDI? And we'll talk about the protocol of MIDI and some examples which means I may be playing some music which is why I have a keyboard and a microphone It's just a good excuse to show off really isn't it? So who am I? What have I done to deserve to be on this stage or as it should probably be called
The ego slide where the speaker brags about its achievements for 20 minutes Not gonna do that because I haven't done anything Industry veteran by the way just means I'm old Developer and entrepreneur means absolutely nothing it means no one would give me a job so I had to create my own But the important thing is I have a term I mean
This is the only musical instrument you can play without actually touching it Even though if you go to the Museum of Music in Brussels, there is a sign on the theorem that says do not touch Go figure The important thing I guess for the purpose of this is that I do like composing music. I'm just not very good at it
so I Thought okay. I will do what I can which is code So for many many years MIDI has been the standard for musicians to Store their music into sequences and the playback now
MIDI means multiple things to multiple people When you say MIDI, that's not enough MIDI sometimes will mean the serial protocol that goes down the cable between Synthesizers between computers and synthesizers or drum machines or other equipment like that MIDI can also mean the file format and there were three forms of file format
They've been interestingly labeled format zero format one and format two Zero is pretty useless everything's on the one track But it is very easy to pause We file my format one has got every piece of music on a separate track which makes it very easy and many file format
Too used by almost nobody but you've got to know it exists. Some people mean that when they say MIDI And MIDI is also the hardware specification There is a chunk of the spec which says exactly how much current has to go through the cable What sort of tolerance is it has to be how fast it's running and all of that stuff We don't need to worry about we're in the safety of the sandbox of a web browser
so we don't need to worry about any of them, which is lucky because web MIDI is an API for that protocol and Nothing else All the bits that would be useful that are necessary and a sequencer are not available in web MIDI Thank you for coming to my TED talk. Goodbye
It's just an almost nothing in there so if there's nothing in there it will at least be easy to explain Does anyone decide for the message yet? That is a MIDI message If I present it in hex, does that help? No, at least one person's nodding
well The first part of this message is the command 1 4 4 99 and the second one is the channel now a couple of things you can work out from this Hex numbers 4 bits long from 0 to 15. That means there were only 16 channels That's part of the spec. It's been there since the year dot is not going to change
Unfortunately, however MIDI read the MIDI spec version 2 has just been announced So this talk will be completely redundant probably by the time I've finished giving it The first bit is the command and again you'll notice It's a single nibble in hex, which means there are only 16 possible commands It's another caveat. So
two possible things This command Dictates these bytes So serial protocol they'll come in one after the other and it'll your program will see that first byte It says what is this? It's a note on message at which point it understands that the following two bytes will be a note value and a volume
So the obligatory hello world apologies for the use of the word navigator in there, it's probably not really necessary
But this is the thing that will take in a message Passes it out and says well, we know note on is this 90 and we can track it out to the console So to prove that works
Probably shouldn't say that and then do a live demo. I should do the live demo and then so let me clear that If you look onto the bottom window One message for on one message for off still only three bytes each time 64 volume one
Where's the time where's the timing information in that message? See that can anyone tell me where the time is gone The time is in real life All the time in web MIDI is controlled by how much time you wait until you send the next message There is no other way around it So you are forced to rely on JavaScript giving you very precise timing in order to create very precise music
This is why I titled it garlic bread Is anyone aware of the Peter K sketch the stand-up he does where he goes garlic bread Because it's just incongruous you have a MIDI
protocol that is for music that is real-time that needs to have very precise timings and Then you go and stick it in a web browser Which is sandbox and is not very good at precise timing Which makes it a little bit painful at times if you try to do anything live
So let's continue with that so It's if it's a serial protocol. There is one problem with handling it. How do you know if you're in the middle of a message? or not We've established. This is the note on command. This is the note on message Well, what happens if the first byte we read is 60? What command is that?
What's it suddenly going to start doing at that point? What if we come in here? What about a hundred? Well, the clue is by writing it down in binary So if this most significant bit is one it means this is a command Something is happening. Now. You should be paying attention And if the most significant bit is zero, it means this is the data for a previous command
Very simple to pause. It's one operation and present 0x80 But I just said there were 16 possibilities for this number if we're using the most significant bit Then actually we only have eight possible commands and that's it. The entire specification is just eight commands
And that and that that's these ones so note off is the most important one because it's the first one No idea why note on is the second one Although you will find and this is where it gets fun
that hundred if First byte is the command the second two bytes of the data bytes now Any number of data bytes you like they're controlled by the command This is the volume Some keyboard manufacturers have decided that if you put zero as the volume on a note on Message that means note off
Because obviously it's logical, right? So you've got no on a note off the key pressure just for hitting it harder essentially Parameters and programs for changing the sound because although there were no sounds in the MIDI spec and there were no mid sounds in the MIDI files There is still the indication that there would be a sound so you say I'm just telling you the note
I'm saying this note goes on this note goes off I'm not telling any more about it But I will tell you I would like the synthesizer to play a piano or a string sound Doesn't tell you what the string sound is or how the string sound is formed. Just something in the general area of strings
Changing the pressure more after touch thing the pitch wheel obviously this expensive keyboard does not have pitch wheel But if you do then there was a special message for it and then there's the meta events for everything else It's like they started designing the keyboard. They thought yeah, we need notes on we need the ability to change the program We need the pitch wheel. Oh, dear. We've run out
So everything else is lumbered in this bottom stage And there are just hundreds of these meta events system s exclusive messages that send patches Just full of them This one. I know it's it. I know it's dot H file because this is that this list comes from my original C version of this library And it just lists them all in there
So what is message what we're missing in web MIDI. We've established the timing message. It is missing That is just something you have to time yourself When you deal with MIDI files, and I'll show you some bits later You have to specify that time so you have to count the time yourself You say this note comes on then this note comes on and there is half a second between it you then write that in
The duration again, if you don't have timing messages, you can't have duration So you have to handle that yourself think about how long each of these notes are going to be Cords Cords are kind of fundamental music
This is harmony every single piece of reasonably good music will have harmony and quite a lot of bad music will have harmony as well But there is no way of describing a chord You have to say this note on then this note on and then this note on At the same time or as close to the same time as you can possibly manage in your code
Ideally, you'd want that time to be zero though. At least shorter than 10 milliseconds And there were no conventions. So if you use one of the system exclusive messages, it's up to you How you decide to handle that? Every keyboard manufacturer has their own set of things that they put into those messages And if you want to process them or understand what that keyboard is sending you're on your own
so we've got the protocol is a stream of messages and I was going to give some examples, but I Will have to go out of here for that So we shut that down
This is the one that I've been fiddling with for the time. It's This one and I put this up as loud as I can and then flip this on
Now you can tell that there is a little bit of a latency on me pressing the keys and hearing the notes which is unfortunate But that's unfortunately nothing of my doing I'm not that bad a coder It is just the time that it takes For the message to come through and the browser to react and the sound engine inside the web audio engine to play those sounds
Because there isn't anything much is really doing there That's a web audio backend being triggered from from the from the JavaScript in the browser Now if I that's not good this was just an example this has got nothing to do it this is an example I was playing with earlier. I was curious about waveforms and how waveforms are created. So I built this web audio thing to
Create waves using different harmonics. I just needed time to hook it up to the setup today But if I show you another one like this soundboard, for example, if I play notes on here now
I didn't get the looping quite right on it, but All I've done here is I've listened for the note down message and when I get that note on message I trigger the sample loop. I don't do anything else when the note up comes. I don't stop it when the second note down comes I
Stop it. That's logic. That's one variable. That's one check which you can have hours of fun with Now while I'm doing that you might hear something else going on
There's a bass sound going on under sit underneath each of those samples that bass sound the bass sound from that window
When you open web MIDI and you say I would like to do some web MIDI, please mr Web browser and the web browser goes sure which would you like and it will allow you to enumerate Every MIDI device plugged into your machine Which is great. It means if you've got multiple keyboards, you can have one keyboard doing this thing and monkey They're all doing that thing works brilliantly
Unfortunately, it works brilliantly for everyone else as well So if you happen to have two apps that were running in different browsers, it means you can see the other person's traffic I Say this because I'm using web MIDI for music stuff. Mostly. Well as much as you can call that music. It's just a loop
But because I'm using if we say I think very much in the music sense, but these things are cheap, right? It's a ten of anyone's money There is no reason why you can't take that apart use the switches and hook them up to something else You might like to do a game controller, which is more accessible than traditional game pads That would make a very cheap and easy accessible game pad controller and it's controllable through the web browser with web MIDI
Unfortunately, if you were doing anything sensitive like you are requiring someone to put in a passcode using those controllers it would also be visible in another window and Currently as it stands
There is nothing to stop you It will be visible in every window and you can't stop you can't say oh I want this for myself That controller is available to everyone and you can start that up on your web page without a click Without the user having to perform things if you've done any kind of web audio things in the browser You'll have probably noticed that it doesn't work until the user had actually clicked on something
It requires the user to interact with the page before it gives you control or for web audio device Not the same for web MIDI you can just look at that traffic at any time another example we had which is probably the interesting one I suppose is
That one I think and of course it's still running in my other window, so let's shut that down
Oh Which is about the closest I'm gonna get so I'll leave that there or quit while I'm ahead Put that back on Now this is obviously a proper example of a base sense, which I didn't write but it is quite fun and is it is again using web audio and Web audio as I say you've got to click here before you can actually make sounds
But it's a nice example, and you can see that it is possible to build quite big and fun things and at this point I realize the person who's written this might be in the room No good That would be awful it's happened before actually sort of mid lecture activity thing I was at a computer thing
Funnily enough me computer thing who'd have thunk it Was it a computer thing and I was we're talking about old computers as my friends often do and I happen to mention this trick on the zx81 about how you could put components across the chip to fix the square root bug that existed on the zx81 at the time and I was explaining how it worked and why only worked in
That case and then I found out the guys that next to me built it And you know the mansplaining level was a 10 out of 10 at that point. I mean fanboy was off the charts But it's what it's one of those things. I'm sure I'm going to give this talk and the person who wrote it's gonna be here Which is a shame because it's really nice. I mean that's
And it works real time so I remember back in the 90s. You could make a whole album with just that I
Don't think I should have done that let's do that instead. Oh so guy and we've got some library links, so So as I saying I use this for music
But you could create a game controller by ripping out the innards and putting that into you know some sort of Something else as we say it's not keyboards drum pads They're very very fun. They're very easy I like those sort of things the Novation long pad which I can't quite afford but they are very pretty and I have lots of lights on them and
You could do it can control lighting rigs When I did some theater stuff a while back the lighting rig that was a big desk, but controlled by MIDI same protocol note on meant light on Note off meant light off But it's it's doable. It's it's it's a standard protocol that seems to have gone beyond what it should have ever been doing
The rate of MIDI is very slow. It's about 33100 valve rate essentially Well, she's slow when you think of how many notes could be playing in a piece of music Even a fairly straightforward Symphony with only maybe 50 or 60 players
There's 50 60 notes in their minimum You've only got 16 channels and you've got multiple notes So you're probably not going to get all of them playing at the same time Because you can't play all of them with zero time gap. There will always be a time gap between each note So it'll never be quite right This is the the slide you might want to take pictures off if you want to go and play with that synthy thing later
also, there is No file handling. Oh, sorry. Did I not did I go? For those that want to take the picture I'm ready for my close-up
So file handling is missing if you want to save the MIDI file You can't if you've generated something nice. You've got to save it yourself. Don't worry. I've written a library for that There'll be a link later for as well The Sysex abstraction. So any as I say any manufacturer such as Korg and Yamaha They will have their own system exclusive data
Which that you just have to go and find out for them and any actual sound the actual noises themselves not in the spec It's just an indication that there is a piano sound here If you do want to have a look my MIDI library is the second one. These other things are also perfectly valid MIDI libraries
My one was based on the C++ library that I wrote many many many years ago I think my original MIDI library is actually older than this conference and there's not many things that are So those were available if you've got the pictures now. Yep So my sort of basic conclusions is the web MIDI is really interesting for input
Lots of things support it not just the keyboards the drum machines and all that sort of stuff Very good for handling. You can build your own but the output is more interesting Because now the browser can control synthesizer rigs and it can control drum machines But the number of people wanting to do that is probably even fewer they want to do input
So with that I shall leave it open the last five minutes for questions, I should update my FOSDEM scorecard That's now the 12th different talk I've given and I still haven't got good at it So we've got a microphone if there's questions
So it kind of looks like someone took a naive translation of a wire protocol and dumped it straight into JavaScript without any kind of design process Was there actually a design process or did someone manage to persuade the Chrome team just to put in this API that no one had really
thought about I don't know Why or how web MIDI came to be I don't know the people I suspect that looking at it They said it's not so much dumping the wire protocol, but dumping what every other MIDI library does Every other MIDI library just gives you that raw data. Here is the stream Partly because you can't process the SYSX messages
There's no simple way of abstracting that other than say it's a SYSX message and here is the binary blob There could have been abstractions. They could have created an API which said on You know with with an event and you say I want this function called when I get a note on event But they didn't do that
I think because they wanted to maintain a direct relationship with all the Preceding libraries and the way that people wrote MIDI applications But that's my conjecture Yes, my library as attached off off the github there does abstract away a lot of this ugliness And it handles that odd stupid case as well
Where if the note is on but the volume is zero it actually means note off Which is one of those things that Fox me for ages There's also the odd thing as well in the MIDI spec that if you do a note on message and then another note On message you don't have to say note on again. You can just keep going So you don't get that most significant bit being one
but for the next message So you just see note on is a three byte message. Why am I getting five bytes? That's just because there's that quirk that was in the original MIDI spec because it will cut down on the bandwidth used You don't have to send a byte and back in 1982 when this MIDI was created that one byte was important
Now it's not but we're stuck with it So we've got these little caveats in the original spec that carried through to web MIDI. I have a microphone for you
So how stable is the web MIDI stuff could it be used for editor librarians for older? synthesizers like I have Some old stuff that I need to use an Atari emulator to actually edit stuff on could maybe be used to reliably do that
You think is it stable enough so is web MIDI stable enough to use as a librarian tool and so forth it is stable enough Web MIDI as a spec isn't gonna be changing. The API is not gonna be changing There will be upgrades for web 2.0 which is coming through the pipeline now
But it's more the case of do you want to spend that amount of time writing a librarian tool When there might be better ways of doing it You it would probably be good for you It would be good for me some of the time But because it's coming quite late into the game I'm concerned that it might only be used for fun projects like this
Rather than serious projects that have already got librarian tools and other system tools But I'd like to see it more done more I know how soon as someone puts a VR headset down. I know that interest is going there and not here
You still have two minutes of interest. Okay, so can we muster two minutes of interest? No, no Thank you a lot gracefully. Thank you very much. Thank you