My talk is about free software and free culture, making music with GNU-Linux. My name is Tobias Platten, I'm a fellow of the fellowship of the Free Software Foundation Europe.

First I'll explain what free software is. Free software is software that comes with four freedoms.

The first freedom is the freedom to use the software for any purpose. The second freedom is the freedom to study the work and make changes.

The third freedom is the freedom to distribute the work to others. The fourth freedom is the freedom to distribute modified works. If one of those freedoms is missing, the software is non-free.

Freedom requires that you have access to the source code of the software. But that's not enough, there are threats like devastation, this is the contrast to open source. And there's already the phenomenon of free culture and open source culture.

And there's a difference if something is open source, it is not really free. And in Alex Leavitt's book about open source culture,

he discusses the mass multimedia sharing phenomenon that has built around Hatsune Miku. She is a Japanese pop star, actually a singing synthesis instrument

that is used as a new musical instrument but the software itself is not free. But those media sharing appears in a way that is similar to free. And with free culture you have several freedoms

and media companies try to restrict what you can. For example Vocaloid restricts that you cannot use the sound samples with your own synthesizer. And with free culture you have the freedom, you can change the sound samples.

Now I go to how do you make music with GNU Linux. There are special audio distributions for GNU Linux, some of them are fully free.

Then you need recording hardware that works with your ALSA kernel and not every kind of hardware is supported on GNU Linux but many class compliant devices are.

Recording hardware can be a microphone or a MIDI keyboard. And if you have the hardware you need to understand how to use the software. On GNU Linux there is a sound system called Jack which allows you to interconnect multiple applications

and control them with a single transport. To make actual sound if you don't use physical instruments you need synthesizers or XAMPLUS.

And there are many free synthesizers and XAMPLUS available. Once you have that you can produce your music. I'll explain later how this is done. And once the music is composed you can post-process the music

using different plugins such as LV2 where one can change, can add effects such as reverb or equalizing voices. And in the last step one can export the song as a WAV file

or as an awkward and upload to the internet. Digital music has source code like software and there are several ways how software can be distributed.

For example music in source code can be distributed. Lilypont is a notation program which allows you to enter text and get musical notations.

MIDI is a format that is used by hardware music synthesizers as well as by software. And Lilypont can export MIDI. There is also MusicXML which is a new XML format

which maps all features that can be used in graphical musical notation. For blind musicians it's BRILE which also has software support. And then there is the classical printed sheet music

which also acts as a human readable source code. It can be transcribed in many other formats. For singing synthesis there are special formats like the VSQ format from Vocaloid

which is available as an XML format. It is a proprietary music source code format. The UST from UTAU which is similar. And recently I started implementing a JSON based format

for monophonic music synthesis. These formats can be used to create waveforms which are actually not source code except when you record directly from a physical instrument

then there is source audio which can be processed using different plugins. When you start making music the first step is that you compose the music using MIDI

because MIDI is what most programs do understand. MIDI is the common source code for digital music which was invented about 30 years ago and it can be used to modify musical pieces.

If you have only an MPSV you cannot do modifications as easily as you can with a MIDI. The MIDI format is an open standard by digital instrument makers.

For example Yamaha keyboards have a floppy disk drive or had a floppy disk drive 10 years ago. Modern keyboards have USB stick support and these keyboards can actually save performances on a floppy disk

and then replay the performance. The sheet music can be typeset with LilyPond. Once you have a MIDI you can use MIDI editors such as Rosegarden to export LilyPond

and in Rosegarden you can also do graphical note input either using piano roll or the classical musical score. You can use keyboards as an input

but then you have to quantize the notes and there are also plugins that can manipulate MIDI data with alpha 2 so you can build an arpeggiator

or some other MIDI effects. You can create background chords on your detected melody and those MIDI files are small. They fit on a floppy disk and can be easily shared through the internet.

This is an example of such a MIDI based audio workstation where you can compose songs. In this case it does not detect the correct scale

and in MIDI you cannot really specify this in a good way. This is one problem of MIDI music format but you can manually select a scale for a segment

and then you get these grey lines which are used for the chord of the first tone of your scale. Once you have composed a melody you will create an audio data while composing.

This is actually played live if you run the plugin. There are different ways to do this. With the MIDI format

the MIDI format controls hardware and software sound generators and two examples of these sound generators that I often use are Qzund, that is a Soundfront based sampler.

Soundfront is an industry standard for instrument sounds and with Synet FX you have more synthetic sounds. It has an FM synthesizer, it can create organ sound,

the pad synth is a special algorithm for creating pad sounds and this is event based control so when you press a key on the keyboard an event is sent to the synthesizer and the sound is created.

The most important MIDI messages are note on, note off and modulation so you can change the pitch with the wheel. There are special controller events such as a sustain pedal and portamento time that you can specify

and there are special MIDI instruments like laptop keyboards which can be configured with SYSX messages. You can also use SYSX messages to transfer new sound fonts to hardware instruments.

Here these are two examples. The first is the Qzund. It has a built-in FX rack where you can specify reverb and promise but only globally, not per channel.

And the second one is Synet FX which is a more complex synthesizer where one can configure different sounds and create your own instruments very easily.

When you want to mix different instruments so you can change loudness for different instruments and add special effects such as a compressor

for example if one instrument is not very loud and you don't want to make the instrument globally louder then you can use a compressor. You should always add a reverb to voices

and this has a built-in mixer. It supports both MIDI and audio tracks. For MIDI tracks you always need to instantiate a synthesizer

and this allows you to use the track transport. So when you click play, other sequencer applications will also play. And one of these other applications is Hydrogen.

Hydrogen is a drum computer and those programs are synchronized. They have a common time base and with those two programs which allow you to compose MIDI

you can actually compose songs. What I also do is using singing synthesis that is very popular in Japan and most programs that allow singing synthesis are non-free

therefore I began writing a free replacement in 2011 and at the current time I have a working tech demo also an internet page where one can upload a MIDI file

and get a singing synthesis example. The program supports both German and English singing. It is derived from the free software Speech Synthesizer eSpeak

and once you have that you can master your music and mix all together. Therefore you should use an integrated database that supports plugins.

I already mentioned it. For each singer you usually use a chorus and a reverb but also for many other instruments. A feedback delay line produces an echo effect.

This is often used for guitars. You can use tube amplifier emulation. Compressors can be used to change the dynamics and there are already graph plugins

that provide graphical equalizers and much more. And if you want to have a Kraftwerk sound like you should use vocoders so you can mix an instrument signal with a voice.

In the last step I'll explain how one can publish the music. SoundCloud is very popular but I don't use it

because it uses non free JavaScript. I have my own media goblin instance. There are also public media goblin instances where one can make songs available to the public.

You can sell self-brand CDs to your fan base. For example many Vocaloid producers sell their CDs at anime conventions. And this is the least restrictive way

compared to ways such as iTunes which only works on non free operating systems and requires non free Euler's. And additionally you can use FLAC or OcVorbis if you distribute music online for a different channel and not iTunes.

FLAC is for example supported by CD Baby. And you can also allow remixes by publishing source code. For example MIDI and OSTJ. Many UTAU producers do this already.

And currently there is a new payment system in development called Gnuthala which could be also used by a music store that allows anonymous payments and you get then a FLAC file for download.

And last but not least there is already a Vaweertungsgesellschaft under German law called C3S which tries to make fund free culture

and this is because of GEMA is doing censorship on the internet. And we actually we don't want censorship and we want ways that people who create free culture

can be also supported financially. This is now the end of my talk and you can now ask.

Yes, I have many songs on my hard disk.

So this character here is Hatsune Miku which I showed again. She was created by Kriptom Future Media and I can demonstrate some songs, yes.

This is a song that was created that I found at the site of the Friesoftabule.

This is a synthesizer that was written by a person from Germany and I am currently working on improving the synthesizer. This was Istvig, yes.

And there is already V-Connect that is a Japanese synthesizer that is less known here in Germany.

And this one actually uses recorded sounds and changes the pitch and the length of segments and concatenates the segments and it has been released under GNU General Public License version 3.

It already ran only on Windows. I took the program and ported it to GNU Linux and now I am going to publish the package on mentors.dabian.net so that anybody can install the program.

And the program internally uses the Vorbis format to compress the residual part of the sound. It uses speech analysis frameworks which detects the fundamental frequency and then it detects the spectral envelope

and there is a third component that is called the residual and that part can be only compressed using Vorbis because other compression formats for sound that are commonly used

don't include the phase of the sound and that can be done if you use Vorbis. And in the new version AppPeriodicy is instead used and you don't need the Vorbis format which was actually not designed for speech synthesis.

And one week ago I experimented with the new version of the software and I found out that this way also compresses the speech from the PCM to half of the size but you don't get the same speech waveforms

and it sounds nearly similar to the original sound.

Yes, you create a MIDI file with Rose Garden you can demonstrate it

and then you can upload

there is a lyrics editor you export a MIDI file

and then you send this to the page

this may take time

and when it has been synthesized then the page returns and you get a download link

and it should work with MIDI files that you have created with Rose Garden and you can also use eSpeaks phonetic notation especially if you use English words

Oh, my server has crashed I can't fix that problem currently I have to look into my log files

and find out why it did not work but I had tested it last week and it had worked at the time

the software is public yes, it is published on GitHub that it is public on the web and if it doesn't work you can complain to the webmaster of the server

I'll add an e-mail link where one can complain if something doesn't work and the other thing we can extend

that is also published but its documentation is only in Japanese so there is much more work to do to make the software actually usable

the site is actually econteryx.moe

moe is a Japanese term that is a new domain that was recently added to the domain name system and you can register such domains

because of the algorithms

you can put in the domain but I have found that somebody has used the Ambrola synthesizer for singing but Ambrola itself is non-free and I think it is time that one builds a free replacement

which also requires building voice banks and that is the much harder part than building the synthesis software alone

for singing synthesis, yes for example for the original

this voice is Hatsune Miku

she is sold by Creptone Future Media this one is by Hatsune Miku

and on YouTube there is a reconnect version which was covered but instead free software was used for singing synthesis and it sounds really similar to Vocaloid

but unfortunately this free software only supports Japanese it is not actively maintained anymore and it is much more work to support other languages as Japanese is quite easy to support

in a singing synthesizer and for European languages you have to currently use eSpeak or at least the eSpeak front-end which needs to be refactored

if you use a different back-end synthesis engine there are some more people working on Vocaloid replacements that I have released as free software but most of them only support Japanese and Chinese

as those engines are UTAU compatible and I am currently doing something that is more powerful

my is QTAU which I actually maintain that is the graphical editor I can demonstrate my editor

I don't know if it starts if the jack sound system is not running

now I have to start jack first

and here you can draw notes enter lyrics

and you get a phonetic transcription this is already done so it supports English you can also switch to other languages supported by eSpeak and it works in most cases when a word has multiple syllables

but sometimes it fails and you have to specify manually the phonetic transcription which is a bit hard then you need to read the eSpeak and I did not publish this version

yet there are too many bugs I want to fix and when I have fixed them I maybe will release the program

it doesn't save media it uses its own tracing format but it can import and export media it uses UTAU tracing sequence text files

it was originally used the UTAU format but that format is not extensible not well documented so I defined a new tracing format which is easy to understand and actually Qt has its own tracing parser

and parsing UTAU files that is a text based format actually a modified windows ini that does not work very well and it seems that Qtau is already connected

and it should also produce a sound

if I click play it first renders the sound it takes a little bit

when you do this first but once a note is cached it will be it seems to be a little bit buggy

I made a mistake but it worked perfectly some weeks or months ago before I went to Anime Marathon and after that I changed the format to Json and now it seems to produce invalid MIDI files

as eSpeak has special format for its format so I don't plan to release the program

until most bugs are fixed and it has been tested and some complete chunks have been produced and I'll do a release