It's so quiet. Let's make music.
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 65 | |
| Author | ||
| License | CC Attribution - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/37611 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language | ||
| Producer |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
Ruby Conference 201440 / 65
2
3
4
6
9
11
14
17
18
19
20
25
27
29
30
32
34
35
39
40
46
47
50
53
55
56
58
61
63
00:00
Twitter
00:27
TwitterMusical ensembleComputer animationXML
01:06
AnalogyVibrationAnalogyQuicksortVibrationAnalytic continuationTouchscreenMultiplication signInfinitySampling (statistics)Programmer (hardware)XMLComputer animationDiagram
01:57
Menu (computing)InfinitySampling (statistics)NeuroinformatikComputer animation
02:32
Sample (statistics)Discrete groupAnalogySampling (statistics)Multiplication signQuicksortEmulatorNatural numberPoint (geometry)ApproximationDiagram
03:07
NumberBit rateSample (statistics)MathematicsSineNumberTerm (mathematics)Sampling (statistics)2 (number)OscillationComputer animation
03:45
Sample (statistics)Order of magnitude2 (number)WeightNumberSampling (statistics)Bit rateForm (programming)Fundamental theorem of algebraPhysical systemNeuroinformatikPlastikkarteWaveSound card
04:12
WaveformWaveNumberMathematicsMereologyComputer animationXMLUML
04:38
FrequencyMathematicsLevel (video gaming)NumberFunctional (mathematics)WaveSymbol table1 (number)Sign (mathematics)Cartesian coordinate systemFrequency
05:02
FrequencyFrequencyTerm (mathematics)Set (mathematics)NumberQuicksortFunction (mathematics)Sampling (statistics)DiagramUML
05:33
Bit rateSample (statistics)FrequencyNumberSampling (statistics)Function (mathematics)QuicksortScaling (geometry)FrequencyKey (cryptography)Range (statistics)Context awarenessUML
06:20
Speech synthesisFrequencyRange (statistics)QuicksortMusical ensembleNumberSampling (statistics)Functional (mathematics)Word40 (number)2 (number)Lecture/Conference
06:55
SineGraph (mathematics)Sampling (statistics)Multiplication sign40 (number)Sign (mathematics)Functional (mathematics)Scaling (geometry)WaveOscillation2 (number)Diagram
07:28
Square numberFrequencyForm (programming)Maxima and minima2 (number)WaveformLine (geometry)WaveMathematicsWeightTriangleComputer animation
08:00
TriangleWaveSawtooth waveRandomizationNoise (electronics)Computer animation
08:28
WaveNoise (electronics)UMLXMLComputer animation
08:53
Set (mathematics)Visualization (computer graphics)Structural loadPresentation of a groupComputer animation
09:18
Structural loadWaveRandomizationNoise (electronics)Server (computing)TriangleAdditionWeightSign (mathematics)FrequencyBitMultiplicationQuicksortGreatest elementEvent horizonMixed realityOperator (mathematics)OctaveSineComputer animation
11:40
Process (computing)Logic synthesisBitMathematical analysis40 (number)Heat transferXMLUMLComputer animation
12:09
Singuläres IntegralFourier seriesComplex (psychology)MathematicsFourier transformProcess (computing)Complex (psychology)MassQuicksortFundamental theorem of algebraComputer animation
12:38
View (database)FrequencyQuicksortWaveAsynchronous Transfer ModeTime domainFourier seriesPoint (geometry)WaveformMultiplication signView (database)MultiplicationTransformation (genetics)WeightForm (programming)Computer animation
13:16
Universe (mathematics)FrequencyWaveformSet (mathematics)Sign (mathematics)Noise (electronics)Form (programming)FrequencyAdditionWebsiteWeightTheorySingle-precision floating-point formatGoodness of fitInformationHeat transferComa BerenicesAuthorizationMultiplication signWaveSineFourier seriesWaveformDomain nameComputer animation
14:17
FrequencyWaveRight angleSineWaveformSign (mathematics)SummierbarkeitNoise (electronics)BitAdditionEngineering drawingDiagram
14:48
AdditionBitWaveNoise (electronics)SineMathematical analysisCartesian coordinate systemSign (mathematics)Multiplication signDistance
15:18
Mathematical analysisVisualization (computer graphics)WaveEqualiser (mathematics)Cartesian coordinate systemFourier transformVisualization (computer graphics)QuicksortMathematical analysisFrequencyFigurate numberEqualiser (mathematics)Multiplication signTrailWaveTime domain1 (number)TV-KarteWaveformSystem callMathematicsMatching (graph theory)Asynchronous Transfer ModeForm (programming)Transformation (genetics)LeakRight angleSampling (music)UMLComputer animation
16:43
QuicksortGraph (mathematics)SineMathematicsWaveFrequencySign (mathematics)Domain nameComputer animation
17:36
MIDISineQuicksortDomain nameWaveAsynchronous Transfer ModeFrequencyTime domainMultiplication signDifferent (Kate Ryan album)AdditionComputer animation
18:19
BitSound cardInformationQuicksortNumberMultiplication signLibrary (computing)Computer animation
18:52
Wrapper (data mining)MathematicsWaveWritingSineFile formatInformationMiniDiscWeightComputer fileRight angleBuffer solution2 (number)CodeSampling (statistics)Library (computing)Sign (mathematics)FeedbackSoftware testingWaveSound cardFunctional (mathematics)Physical lawReal-time operating systemGreatest elementAbstractionQuicksortMusical ensembleOperational amplifierUMLXML
19:53
Wrapper (data mining)Library (computing)Real-time operating systemAbstractionKeyboard shortcutSign (mathematics)Line (geometry)Message passingStreaming mediaProcess (computing)Frame problemMultiplication signLevel (video gaming)Computer animationUML
20:22
SineWaveFrequencyTriangleStreaming mediaThread (computing)Loop (music)FrequencyWaveMessage passingTriangleFrame problemLambda calculusLine (geometry)Multiplication signSign (mathematics)Computer animation
20:46
TriangleStreaming mediaFrequencyThread (computing)Loop (music)FrequencySampling (statistics)Logic synthesisSound cardNeuroinformatikFilter <Stochastik>Physical systemPlug-in (computing)WritingComputer animation
21:46
Visualization (computer graphics)WritingPlug-in (computing)Different (Kate Ryan album)Video gameFile formatEstimationComputer animation
22:22
Server (computing)Einbettung <Mathematik>Parameter (computer programming)Sample (statistics)Physical systemProcess (computing)Cartesian coordinate systemObject (grammar)MereologyFrame problemVirtual machineRemote procedure callCodeSampling (statistics)Computer animation
22:55
Computer fileCodeStructural loadType theoryGreatest elementConnectivity (graph theory)Musical ensembleServer (computing)Gastropod shellComputer animation
23:19
Library (computing)SimulationCalculusServer (computing)Connectivity (graph theory)Musical ensembleSet (mathematics)Structural loadGreatest elementFrame problemFunctional (mathematics)Service (economics)Process (computing)CodeSign (mathematics)Wave functionBitQuicksortWaveSineTriangleComputer animation
23:57
Library (computing)SineLambda calculusWaveSineBodenwelleSign (mathematics)OctaveComputer animation
24:22
SineLibrary (computing)Lambda calculusSquare numberFrequencyNoise (electronics)Greatest elementSign (mathematics)WaveFunctional (mathematics)Logical constantRight angleMultiplication signPhase transitionXMLUMLComputer animation
24:49
SineSquare number3 (number)Library (computing)Lambda calculusNoiseFrame problemComputer fileOpen setBit rateServer (computing)FrequencyRight angleSineMultiplication signFunctional (mathematics)BitComputer animation
25:15
RandomizationFeedbackFunctional (mathematics)Computer animation
25:39
Square numberSine3 (number)Lambda calculusLambda calculusFunctional (mathematics)Binary multiplierReduction of orderWaveLinearizationMultiplication signSign (mathematics)System call
26:07
Lambda calculusMusical ensembleBitString (computer science)QuicksortWorkstation <Musikinstrument>Sign (mathematics)Multiplication signPotenz <Mathematik>Computer animationXML
26:34
Lambda calculusDrum memoryMultiplication signSineWaveSign (mathematics)Computer animation
27:10
Lambda calculusQuadrilateralString (computer science)Musical ensembleWaveSampling (statistics)Computer animation
27:53
DemosceneArtistic rendering
28:47
Artistic renderingDemosceneSample (statistics)WaveSquare numberSineTriangleNoiseNumberSound cardFunctional (mathematics)Sign (mathematics)WaveSquare numberDifferent (Kate Ryan album)TriangleSolid geometrySineUMLComputer animation
29:23
Mathematical analysisQuicksortProcess (computing)WaveComputer fileSign (mathematics)Musical ensembleLibrary (computing)Plug-in (computing)Computer animation
29:52
Real numberMusical ensembleBlogCodeComputer animation
30:24
SoftwareTwitterComputer programmingMusical ensembleShared memoryComputer animation
Transcript: English(auto-generated)
00:19
OK, so let's get started, because I have a lot to cover,
00:22
and I want to make room for the fun stuff at the end. So my name's Lauren Siegel. I'm on Twitter. Elle Siegel, you can tweet at me, follow me. I tweet about fun stuff. This is just a warning. There's going to be music in this talk, so if that's a problem for you, the door's still open.
00:42
Otherwise, feel free to join us. But what is music? So to me, music is sound and rhythm put together in some interesting way. That's music. So we're going to be looking at sound in this talk, and we're not going to be looking at rhythm, but we're going to replace it with another interesting R
01:02
that we know very well, Ruby, and we're going to be talking about sound and Ruby, and that's going to be this talk. So let's start with sound. In our context, there's sort of two ways to look at sound. The first one is analog, think about analog audio. And so analog audio is all about vibrations.
01:23
Basically, we have a continuous signal. There's somewhere in this room, there's electrical wire running to the speakers that you guys are listening to, and that has an electrical signal in it that is oscillating some voltage, plus voltage, minus voltage, plus voltage, minus voltage,
01:42
drawn on the screen over there, and that is driving the speaker to vibrate back and forth a bunch of times per second, which pushes sound waves out that you guys eventually hear. The other side of audio, and the side that we care about as programmers,
02:02
is the digital audio, because we live in the digital world, and we don't have the benefits of dealing with these infinity electrical signals, and infinite electrical signals, and we have to deal with samples. And so a lot of people, because there's this computer sitting between your speaker system, a lot of people think there's some kind of magic
02:20
involved in the computer to the speakers, kind of the pipeline. So hopefully this talk will sort of explain what is happening in between your computer and your speakers. Effectively, what we're trying to do is try to emulate what happens in the electrical analog world. And so what we do is we create, we get samples in time of the signal
02:42
that we want to emulate, and we try to sort of emulate that signal that we want to create. So it's not a continuous signal, it's just samples of the signal at points in time. And so what we do is we take a bunch of samples, and then we take more samples, and then we take more samples,
03:01
and more samples, and more samples, until we get a really close approximation to that signal that we're trying to create. So that brings us to our very first vocabulary term, which is the sample rate. And the sample rate is really just representing the bunch of numbers, a bunch of numbers that we want to push at our audio, at our speaker system
03:21
for one given second of audio. So a bunch of numbers representing one second of audio. If we had a sample rate of 20, that's 20 samples per second, and you wanted to generate some kind of oscillation for that 20 second, this is what it would look like inside a Ruby, and so you generate a bunch of numbers oscillating positive to negative, positive to negative for 20 samples,
03:42
that would generate one second of audio at 20 samples per second. Now actual sample rates are much higher than 20, they're a couple orders of magnitude higher. We're looking at somewhere between 44,000 to almost 100,000 samples per second. And basically it just means there's more numbers. That's all it is.
04:01
So really, the only thing that sits in between your computer and the speakers is a bunch of numbers. It's just an array of numbers that you're pushing at your sound card. So we sort of have a good, hopefully you guys have an understanding of like, okay, numbers pushing at the sound card, driving the sound card, trying to emulate the signal. How do we actually create those numbers?
04:22
What are those numbers representing? So to understand that, we have to understand the fundamental waves that will create waveforms. And the first fundamental waveform is the sine wave. And so I'm gonna preface this with, there's gonna be some math in this part. You know, math.
04:40
And I'm gonna try to keep it high level. I'm not gonna try to go into details, but there's gonna be some funky numbers, so if you don't really understand it, don't worry about it. I want you guys to understand the concepts, not the math. So this is the function for a sine wave. Don't worry about the pi and all the symbols. The important ones we care about are the frequency and amplitude.
05:01
And so those things basically explain how we scale this sine wave, both in the horizontal and vertical axes. And so amplitude makes it taller, frequency makes those things, those peaks, closer together, and that brings us to our next set of vocabulary terms,
05:21
which is amplitude, which controls loudness, how loud the sound is or how quiet it is, and then frequency, which basically is pitch, how high or how low the pitch is. And then there's this S value, which I sort of skipped over because it's not that important. It's basically our sample rate. Because we are scaling this value
05:40
to a number of samples per second, we wanna scale that mathematical function to have T equals one second equals 44,000 samples. So I mentioned frequency as well. So frequency, higher frequency equals higher pitch, lower frequency equals lower pitch, but what's some context behind that?
06:01
What are the actual numbers? So humans typically hear somewhere, have a range somewhere between 20 hertz to 20,000 hertz. That's the typical range for human hearing. So the range for, if you were to pick any number between that 20 to 20,000, you would have some kind of sound that would be audible to most humans. But to give you more context,
06:22
the middle A note that we sort of, if you've ever played guitar, you've tuned a guitar, this is sort of the frequency that we tune our instruments around. And it's the middle A key. And this is sort of the pitch that I'm speaking at right now, very close to the pitch that you hear music at. Music goes between around like 200, 2000 hertz,
06:43
that kind of range. And obviously there's higher and lower, but most of the sound we hear is around that range. So pick a number between there and you'll probably hear something that sounds like music or sounds. So if we take those numbers, we take the 440, we take the 44,000 samples per second,
07:02
we take an amplitude of one, we get this sine function, which looks like this. And basically it's just a sine function, not too magical. If we got to 44,000 on the X scale, we would see that basically there are 440 oscillations in this sine wave. And that's all it means is that sine wave
07:22
is oscillating top to bottom 440 times within that one second of time. So the next fundamental waveform is the square wave. And the square wave looks pretty much like what it sounds like. It looks like a square. And so basically what you're doing is you're oscillating from the negative maxima
07:41
to the positive maxima to the negative maxima and back again. Again, 440 times per second or whatever your frequency is. Don't worry about the math, it's not that important. And basically if you connected the lines, those lines going up and down don't actually exist. But if you connect them, it would look like a square. And so that's where the name square wave comes from.
08:01
Then we have the triangle wave, which again looks exactly like what it sounds like. It looks like a triangle, goes up and down and up and down and just oscillates. So anything that oscillates is gonna create some kind of sound. And then we have the sawtooth wave, which again looks like what it sounds like. It's very similar to the triangle wave. Instead of going up, it just goes down, down, down.
08:23
Then we have the noise wave, which seems weird to call it a wave. It's really just noise. And it really is just randomness. And it may seem weird to bring this up, but you'll see why I brought this up in the talk later on. So this is another kind of wave you can create. Probably the easiest one you can create.
08:40
So rather than talking about this stuff, let's actually hear some noise. So I have this audio tool open up here that I'm going to load up. That is really, really small. So I might have to bump down my settings here.
09:12
I apologize to the people doing visuals right now because I'm gonna mess with the presentation.
09:20
All right, so hopefully you can see that over here. Oh, that's better. Okay, cool. So I'm gonna load this up. All right, so I'm gonna play one note here. So that's a sine wave.
09:42
Sounds like a sine wave. Basically what happens if I can play a sine wave like that, I could go up one octave and you're gonna see the peaks getting closer together. So you can see how that happens.
10:01
So they get closer together as the pitch goes up. That's basically it. Next we have a square wave. And this is gonna be a little bit more noisy because square waves are a little noisy in general. So you see the same thing happens there.
10:20
Ignore the little noise that you see at the top and the bottom here for now. I'll explain why that's there in a bit. But that's the square wave. Then we have the triangle wave. So that's a triangle wave. And then we have the saw wave, which again is a little noisier, but cool.
10:41
And then we have noise, which is going to look like randomness and going to sound like it too. That's noise. And then you can take these sound waves and you can add them together. So we can take one sine wave and add it to another sine wave. And that really just does the sort of addition of two waves together.
11:01
And you'll see how they sort of inter operate. Let me just move that. So you can see the interplay of these two waves added together. And they sort of, because they have different frequencies, they sort of create this really interesting intermix. Every sound is basically, every sound that you're going to hear in the real world,
11:21
of my voice, some guitar instrument, so any instrument is not just one frequency. It's multiple waves put together. So this kind of thing over here is really just like, that's what sound really looks like. It's the addition of multiple waves put together. So let's jump back into our talk and let's keep going.
11:45
So we talked about audio synthesis, which is basically creating audio from nothing. And we're going to talk a little bit about analysis of audio. And basically analysis of audio is all about transforming audio,
12:01
visualizing it, looking at it. There's one important tool that is used when you're analyzing audio. And I'm not going to talk about it too much, but it is called the Fourier transform. And the Fourier transform is probably like the most important tool that you can use when you're doing audio processing. I'm not going to talk about the math.
12:21
It's actually, it's pretty complex. It's literally complex. It's complex math, so I'm not going to talk too much about it. It would be way too much to talk about here. I do want to talk about the sort of fundamental principles behind it because they're pretty easy to understand and hopefully it'll sort of make things more clear. The way I like to think of the Fourier transform
12:41
is you have this waveform on the left of time domain. This is a square wave going up and down. And the way I like to think of it is you take this square wave at a point in time and you turn it sideways, and you're looking at this two-dimensional view now of this wave, and you're looking at the side view of the wave. And it's sort of every single frequency,
13:01
because I mentioned every waveform has multiple frequencies involved in it. You're sort of looking at every single frequency involved in the creation of that wave on the left. So you're sort of turning it sideways and looking at frequency mode. That's the way I like to think about it, and it makes it easier for me. If you want to learn more, there's a website called thefouriertransform.com.
13:21
It really has a lot of good information about the Fourier transform, really well-named domain. The thing I like the most is there's a quote from the author on that site, which really is a good fundamental explanation of what this theory is. It basically says that every single waveform that is created, that you see,
13:40
that the square wave that we saw, the noise waves that we saw, my voice, every single sound that you hear can be recreated using nothing but sine waves. So nothing but sine waves, so all the other waves that we saw, the square wave can be recreated using nothing but the addition of sine waves. That's a really powerful concept,
14:01
because it means not only can you add a bunch of sine waves to recreate any single sound, it also means you can take any single sound and decompose it into the set of sine waves that created it, and that's really powerful because every sine wave has one frequency, so you can see how that works. So here's an example of that. Here's a waveform on the right
14:22
that we're gonna try to approximate using adding a bunch of sine waves together. So we add in one sine wave on the left. It doesn't look anything like it. Let's add another sine wave. Okay, now it's starting to get closer. We'll add another one, okay, getting closer, and then finally, we add the fourth sine wave, and look at that. It's exactly like the wave on the right,
14:41
and it turns out that this wave on the right was actually created using the exact summation of those four waves. So I told you guys to ignore the noise in the square wave. The reason why was because Ableton Live is actually creating those square waves using the addition of sine waves, and this is why over here that you see the noise
15:01
is because we're actually adding sine waves, and if you don't approximate to infinity, you're gonna see a little bit of noise at the top and bottom where those oscillations happen in the sine waves. So basically what you're doing is you're adding a bunch of sine waves together, and you're approximating a square wave like that. So what are some applications of analysis?
15:21
The first one I like to call passive analysis of waveforms, and that's basically pitch detection, tuning, visualization. Instrument tuners are a really good example of that. You sort of get the frequency mode, sort of look at it, figure out if you're on the right tune,
15:40
otherwise you change your instrument or you retune your instrument. The second one is wave shaping, and that is when you actively change the waveform. So we go into frequency mode, we take this waveform, we turn it into frequency mode, we have all the frequencies that are involved in this audio sample, and then let's say we wanna get rid of the low frequencies,
16:01
so we delete the low frequencies, we get rid of them, and then we take those remaining frequencies and we turn them back into time domain, and now we have this sound that has everything but the low frequencies involved. And that's all that equalization is, is going into frequency mode, manipulating those frequency waves, going back into time domain, and now you have a manipulated sound.
16:21
Auto-tuning works very similarly. It uses pitch tracking to figure out what the actual sound is, and then it sort of manipulates those frequencies that don't match, and then slightly moves the ones that do, and then brings it back into time domain to create an auto-tuned sound. So, I'm not gonna show Fourier Transformer auto-tuned stuff like that.
16:41
That would be awesome, but I don't have enough time. Instead, I'm just gonna show you guys sort of what this looks like. So I have this graph over here. Let me just load up the sine wave that we saw before. Okay, cool.
17:01
And so I'm gonna play one note here. I'm gonna play it really quietly so it doesn't bother you guys, but basically what happens is I'm playing a sine wave note, and you'll notice, again, sine waves have one frequency involved in them. So you'll see that one peak represents that one frequency of the sine wave, and that represents about 900 hertz. It's the A4 note, and it's about at 900 hertz.
17:23
And so I could go up an octave, and then you'll see how the pitch changes in the frequency. So that's basically what it looks like. Now, let's look at a square wave, because I showed you guys a square wave here, and I said it was a bunch of sine waves added together.
17:41
So let's look at what those waves look like in sort of frequency domain mode. So this is the square wave. Let me just, so that is that. I'm gonna not keep it on too loud. Okay, there we go. So that was the square wave. You can see that it's actually the addition of a bunch of different frequencies,
18:01
and those frequencies were created using sine waves. And so that's really what it looks like. And so if I want to get rid of one of those frequencies, I could sort of delete one of those, go back into time domain, I'd get a completely different sound. Cool, so now it's time to get practical.
18:25
So we learned how audio is represented. We got a little bit of information about that. We learned how to create that stuff, those numbers that we have to push at our audio card. And then we looked at sort of how you could process that audio. And now it's time to actually figure out
18:41
how we can use that information to write audio with Ruby. So I'm gonna show you guys three audio libraries that you can use today to create sound using Ruby. The first one is called Ruby Audio. This basically lets you write audio directly to WAV files, .WAV files on disk.
19:02
It's basically a libsoundfile wrapper, and it looks a little something like this if you were to write a sine wave to disk. Don't worry too much about the code here. Basically, we're creating a buffer that contains 4,800,000 samples. We have 4,800,000 samples per second for 10 seconds, 4,800,000 samples.
19:23
And then we fill it with the sine wave that we saw before, that little function we saw before. And then we just write it up to disk at the bottom there. Nothing too crazy. This is a really fun way to sort of write out music, play it, test around with writing functions out and see what they sound like. But it's a really slow feedback loop.
19:41
You have to write it out, you have to play it back. It takes 10 seconds, blah, blah, blah. What we really wanna do is we wanna write directly to our sound card so we can get instant real-time feedback. And so when I was doing this talk, I wanted to get instant feedback on the stuff that I was playing with. So I wrote a library called Easy Audio, which lets you basically write directly to your sound card and hear that stuff in real time.
20:01
It's basically a port audio wrapper, which is, port audio itself is an audio library that is cross-platform, lets you basically, it abstracts out the OS layer when dealing with audio. And there are FFI bindings in Ruby for this as well, so you can get the port audio gem and just use them directly, but the low-level API is really difficult to use
20:21
and it's not fun. So I made Easy Audio and it's basically way easier to use and you can just play a sine wave with that one line. You can open a stream, pass in a lambda that processes one frame of audio at a time, play a sine wave. You could do fun things like taking a triangle wave, starting at a frequency of 220, increase the frequency over time,
20:41
and let me just play that for you real quick. So this is the increased frequency one. I'm gonna play it here. So that's what that sounds like.
21:03
Let's move on. Cool, so we can play with audio, we can do fun stuff live, but there's one problem with writing directly to your sound card is the fact that you don't get to take advantage of all the other fun stuff that your system,
21:21
all the other audio tools that you have on your computer, synthesizers, filters, like delay and gain and all that fun stuff. So you can't really do anything if you're writing directly to your sound card. What we really wanna do is we wanna write to our audio tool like Ableton Live that I was using before so that Ableton Live can also add in its ability to synthesize
21:40
and sample and do all that other fun stuff. So we wanna write an audio plugin, which means we wanna write a VST. That's typically the most common plugin that you'll find out in the wild. There's different kinds of audio plugin formats, but VSTs are definitely by far the most popular. Any audio tool that you'll use, if it supports plugins, will most likely support VSTs.
22:04
So I wrote a VST that will embed Ruby so that I can actually write Ruby in Ableton Live and actually play around with sound in Live, get the benefit of all the tools and visualizations inside of Live.
22:22
So basically what I did was I embedded Ruby inside of a C++ application. Don't worry too much about the code here. The important part is it's basically embedded Ruby, calling out to an external DRB process. So I'm calling out to another process that actually processes the audio locally on the system.
22:41
And then every single frame where the VST's asking me to process a bunch of frames for samples, I basically call out to that remote Ruby object sitting on my local machine. I return back with the samples, and then it returns back to the audio system. Let's see it. So this is where it gets fun.
23:01
This is where we can all have our fun here. I'm gonna load up the VST code here. So I'm gonna go into EasyVST, and I'm gonna type rake. That's gonna drop me into an interactive shell, which basically has the music server loaded up,
23:23
the server side of my component here. And I'm gonna load up a audio, a little set that I created with that VST setup at the bottom here on the left. And so I have the VST loaded up, and I have the server side component loaded up. Let me just show you some of the code here.
23:40
This is the server. I created a bunch of instruments that process sort of one frame of music. So here's the sine wave function. So that's a sine wave function. That's a square wave function. That's a triangle wave function, et cetera. And then there's just a little bit of a music server that actually processes that audio. So what I can do with this music server is I can set a function. So let's play a sine wave.
24:01
And then I can go into live here, and I can actually play one note. Sorry about the pitch there. So there's a pitch. There's some sine waves action going on there. This is a sine wave that's created with Ruby. Same kind of deal with the octaves.
24:20
So I can do that. I can create sine waves. I can create square waves. Warning, this is gonna be a little bit noisy. So that's a square wave. You'll note that we're actually using the exact square wave function. We're not approximating with sine waves. There's no noise at the top and the bottom. We can do other fun stuff.
24:40
So we can add sine waves together. This is three sine waves added together with different frequencies. And so you can see you get the same thing. And we can do really fun stuff. Like we can take the frequency right down here. I don't know if you can see that.
25:00
So this phased constant right here is really this crazy function that actually oscillates the frequency over time with another sine function. So it's actually oscillating the frequency over time. So this is gonna sound a little bit crazy. Bear with me here. Let us, cool, I did that.
25:28
So you can do fun stuff like that. You can play around with Ruby functions. Create interesting functions, random things, do all this weird stuff. And you get the feedback live in Ruby. It's really fun to play around with all this stuff.
25:40
So you can do all that fun stuff. We can also do real things with sound. We can actually create real sounds. So I'm gonna create a stabby lambda here and make a function that calls a sine wave and then multiplies the amplitude by a fall off. So I'm gonna use a linear fall off here. And this linear fall off is basically going
26:02
to make the amplitude reduce over time linearly. So I'm gonna play that for you guys. So we can do that kind of stuff. And then we can actually make it fall off exponentially. So we can make that sound like this. Let me just bump that up a bit.
26:28
So you can do fun stuff. That sounds like maybe like a string instrument. So we're getting close to sort of instrumentation here. Cool, so we're getting closer to making music. Let's create some cooler instruments. Let's create a bass drum.
26:41
So what's a bass drum if not just a sine wave times the exponential fall off thing. So let's go back to that. I don't have to change code. It actually is the same thing because, so this doesn't sound like a bass drum. But that does.
27:00
So a bass drum is really just a sine wave that has a really low frequency. And so that's our bass drum. So let's create a snare drum. Okay, so a snare drum is really just noise. So let me bump that up.
27:25
Cool, so we have a snare drum, we have a bass drum, we have a sine wave playing like a string instrument, we have a bunch of cool stuff. And so now you guys have the tools to create music with Ruby. But it would be unfair of me to get you guys all the way here and not actually create music with Ruby.
27:42
So I created a song for you guys to show off what is possible. And I hope you guys enjoy this song that I created using nothing but the Ruby samples that I talked about in this talk and I showed you guys. So hopefully it's not too loud. Here we go.
28:40
That sound using nothing but Ruby. I'm just gonna close off this talk with a little summary here. Basically, audio is not scary. It's just a bunch of numbers that you're throwing at your sound card. You're just telling it to vibrate a speaker system.
29:03
That's all you're doing. You're just throwing a bunch of numbers at a sound card. So we learned how sound is represented. It's just a bunch of numbers. We also learned what those numbers actually should be, like how to actually create those numbers using sine waves and different functions that oscillate, like sine waves, square waves,
29:21
solid waves, triangle waves. We also looked at how you can use Fourier transforms to sort of process and analyze audio, as well as do funky things with those sine waves. And then finally, we took that knowledge and put it all together and actually applied it to Ruby
29:40
using three libraries, one that lets you write out two away files, one that lets you write directly your sound card, and then one that actually acts as a VST plugin that you can use to plug into other sound tools. And then finally, we made actual real music with Ruby. So I blog about stuff, various things.
30:01
Lately, I've been blogging about music, so if you want to read more about all the music stuff that I've been digging into, you can go to my blog. I have a couple posts up there. I'm gonna be adding a couple posts later on with all the stuff that I talked about in this talk and more. All the code is available on GitHub, so you can go check out Easy Audio and Easy VST and play around with them and do fun stuff with them.
30:24
Again, I'm on Twitter. You can tweet at me, follow me, ask me questions, et cetera. And again, I just want to say thank you. I have two passions. One of them is programming and the other one is music, so I really enjoyed the fact that I was able to pitch a talk at RubyConf that included both of my passions
30:43
and it actually got accepted. So thank you guys for actually coming here and letting me play music for you guys and letting me share my two passions. So thanks.