please log come June influenced I'm so given the woman pals how

tolerant a terrible mistake I t the topic of a citizen of that this is I want to enzymes they defined introduction yeah

I'm really happy to be a very excited actually have to put that may be delivered to excited so please bear with me but today I talk about the

evolution and design of digital music instruments but the probably not so technical but by might the preposition what to inspire makers to like pursue this field and join me in my passions for for these instruments because there amazing amazing

itself to through very briefly about me

a fungal in I'm on I'm for something Germany I studied in which just that the interaction music technology and run all I'm running a

little is a tech start-up company in Hong Kong as the CTO and I'm a drummer but 1st of all where do lies vision music

instruments come from they actually have a really long history despite what you might might think so you know because a goat music music instruments like the piano the valine they all like this centuries-old than ever refined and refined and they're really very good instruments but that the verge of the 20th century people discover new technologies and and I wanted to utilize this in the context of music to

already in 18 100 97 was

the on built which was and of a lady huge organ kind of thing and it's that the idea was there was no radio so they want to dispute these sounds to the living rooms of the people so I was like the USA cable thing we wait like to atomic to other towns as estimating and and it was the first one that someone spends all 14 on then

the 1st Oracle open-air controller because you don't touch anything

is is some form is built by Leo Leon turning all if there is real name and is I think is no sound coming out of my

that with this is a little bit you in terms my no it from the X files for example that we used a wizard a then 1928

the mission of the new builds the owned but the but which had like a unique control that you see this ring which was like on a slide so you could like have met have like super nice a general effects and they're very slight the control of of dynamics so of this sounds like who like this and and Radiohead

actually also rebuild 1 recently and use it a lot of a lot of songs because is really amazing instrument all those early music instruments they had to buy a they were they a of approval by society just because it was a futurist time but the same also is

is that for the total new which is a predecessor of the synthesizer and had later in in the race's resisting why are we playing play some leaders in a room with a and I'm and a and later the Hammond

organ was invented but it's not doesn't reclassified as an electronic instrument because actually electromagnet mechanic but it is a such important mention that as the 1 to show it because it was a very beneficial for the whole field and

later the electonic sec but was invented which was the 1st Israeli which was like the incident and that was 1 voltage-controlled and had like extreme precise control like a controlled off of temporal properties of the sound so like this

and the new new age but answers which

is computer music for in

57 max Matthews at Bell Labs but use the the or the telephone and network and the computers that it if is there had to actually build the 1st computer program for sound synthesis and at same time also the remark to form of Z was was of poem published all that you've but invented soul but it's also think

this is 1 of the 1st program synthesizers or I for the know then a very popular moves and then to demote in the minimal

and we charged under the as before famous for the fat sound and they're very like huge commercial success then another new became

digitalized sold the fellow I was 1 of the 1st digital digital synthesizers and the media media was invented or specified as a as well as at this occasion so you could actually now connects synthesizer stream to to another to come to use actually this is very important because now the interface was separated from the sound source but it could just send control signals which is the problem is this in in a minute and I also 1980 was like the 1st popular popular that's a unconventional controls

the lake get away from the whole geval and a lot of it is sent them a lot of like research institutes and institutions were founded in flextime yeah yeah

come and and so on but the the CMI fallacy that's also this think the my then so

the eighties of course as you know we all know this sounds that is the popularization and of synthesizers her soul common use until today in popular music and 1 of the cornerstones of that was the DH 7 phenomena which was the first one that you used FM sciences and also in the mid eighties Millipore kept by invented max max is until the day you use as some some like the most important or 1 of the most important but like computer music but post office for composing but also sound synthesis and and mappings those of us like a

visual programming ID and out of this also pure data was working out of the same invention will and

then controllers get more more unconventional so for example makes my thesis and I were invented computer-music fell but he developed the radio but on there and then they were controllers with like the more advanced sensors and like for example there's also the the lightning which use infrared and sounds kind of knowledge like It's lies dialog on the conventions but very different tickets and in those Judah 9 conference was

for that was the 1st non conference which is until today the biggest conference for new music interfaces and digital music instruments and between 2002 and 2008 a work of 2 and 66 music instruments presented and I don't have data for 2002

to a 2008 until today but but they can see how many new investors there are and how popular this view became in the future I think it will be even more

like that of we have an increasing number of new inventions and great new instruments because we have more accessible ascenders and materials and and also I think the committee of musicians which tends to be a little conservative friend and but I'm not very keen on trying new instruments they will they kind

of open up a little bit is see more and more like a new instruments conveyed by a musicians profession musicians the yep at the same time so I reviewed a lot of patterns in this in this field and devotion this only mentions but they can every known then admitted the commercial even I just normal like public success and that's also give my talk I want inspire more people to build them in a way that they don't like stay Garrett but now to the design the i is composed of the parts

so you got the interface the resistance index for the EU the instrument but this this is very important that you have the interface that is separated from sound source this also a small

a computer and the mapping Napster's input parameters to the output parameters so synthesis parameters but that's pretty amazing because now the mice are

flexible you can customize them you have the completely is fully the open you can do whatever song you want them and you can to read the controls beyond what

is just the physical object that is like but you have to rebuild anyone it into like it's all fast to to to get a nice music expression but my in my present research and

work I focus more on the gesture control science or less on the synthesis side but because I found I find the controllers are really very interesting we use them all the time for example you know you use a keyboard to type its specific if designed for this purpose use game patch game but sometimes even use

with a keyboard for gaming or game that you his if the right and you see something on on the on the console and so they see that they have to be ready for 1 purpose designed for this thing and that's 5 and instruments is as a and there are different types of themselves but I find this very

interesting due to justify them like this so the idea of augmented music instruments have instrument like the minus the instrument Spidey mice an alternate controllers but suffered wonder music

instruments are really instruments where you stick a bunch of sensors for them for example buttons or as as ours borrow 1 of you can imagine actually but he's he also making yourself a false so that the trombone is like a distance measurement of ultrasound and

all those things there are then I met to synthesis that runs like in parallel to the actual instrument of then there are more commercial

type of this of instruments are ties would like all because they 10 of its tried to emulates the the musicians gesture of unknown instrument that perfectly so I imagine

idea you trumpet is a property of automata but you get this year this instrument and you can immediately use because as the sex and gestures but the song can be different and their instrument is by controllers where it's a

little the form of further from the from the instrument like it was you can still use some of your gestures that were used as a musician but it makes more of of energy can for example users of the new instrument the seaboard granted the 1 with a wobbly texture and this like if young nobody compressed down and it has like a somewhat control of of a brother and and they can slide pitch and yet is also the idea hot which is on the top an a right and so the instrument 1 which was the reason very new instruments and an

ultimate control has nothing to do actually with any of of the museums tremendous what was there before but somebody is the year the order cubes which are actually the cues that know about each other's position and and rotation so you can actually imagine like is not is like that but the kind of prevention music instruments of more like for sequence control of of things what I is called an ambient instrument then I for example carry could use of famous but instrument to make so he use everyday objects like but a

mark or shoe because he said it but can relate to that so immediately that they just pick it up and they did not want to put a hold it and if I place the sensors at the right spot then orgies of way faster than if it's just an arbitrary object and but there's also the Office which is also very interesting shape and

light and also like the word for example graphic tablet I like this it's not as as it as an example because it is pretty accurate sends loss and pretty is also an everyday object that you you know how you to use a pan and you very skilled already added 1 and use it as an instrument and

this is also part of the other controllers but average of controllers the a drug you see the true color the tuple to be people play at the same time and directed react table on the left from that you're going go there who where is a tangible interfaces you interact with the digital but digital information on the physical space the all projected from the bottom and it reads whatever objects to place on them and it can rotate them and change the distances between them and make music like this

then a body controllers terms of which are very

popular at the moment and this is well you want to use your body gestures whatever Jesse 1 of the like you want to just and acquire all data what you hand movement or body movement the bowl of an example this mission my speech he has that it's called the hands of the very famous 1 and image and he for example is she is she didn't do look the graphs in the middle on the right is 1 it's the shanty would be tested on the bottom is Marco Donna Rama who uses fire signals but Amazon's directly to make music then also open air

controllers of the controllers that you don't have any or any the and feedback anything it's just like your hands at 11 instances where more the all always computer vision this any and for this in the

future we can all imagine where the where joint goes your augmented reality now until holy immersive systems but to the

interface but how do you design an interface how how do you start doing

this but for me what I'd like to do is I think about gesture of our how does a

musician move what movements to do this you do or what can you imagine what Cook would correspond well to dynamics of volume and to pitch would it be discrete would be it a sliding pitch but and what corresponds to timbre often music

but Tomba by should everyone knows that the term is but is there is to a small shop formal metal no and but was to to get an idea of what you could do is the 1st thing you do is you of Cerf musicians how they play so then you

know that because it's like what it coming from and and they of interesting gestures and on musical instruments from for me for example I but the study where I studied do justice of harmonica players and

have a lot of interesting suggestions like the hand cupping gesture or to some other movie instrument and now that different in these with the fingers and also like angles

and so the whole motion capture study which is amazing to get the lots of data and numerical data their optical ones they act X a skeleton ones and the and also many field motion capture but the problem is that this super expensive we only find a man like research facilities and so what would be better for the for the the winemaker is to use the iOS

just look at musicians perform and use open source of my favorite is that you and you just annotated through the gestures that they make but to say OK but I'm like you job what movement and then the all these things to annotate them and then you also listened to the sound or you can even analyze sound and

exploded as a as a friend some properties for some of us on is sunny visualize it and put it on there that is a great offer for analyzing sound and Thank you compare what what property of the sound changed when a musician did this check this gesture but some all for harmonic I was like people leading backwards when they were when they were more expensive and more high pitched and and those things you can you can use them they already used to those those gestures annotations of extremist-the simply as a site you have

different videos for example on where labeled with 3 there is a is some some data so spectral centroid I think and and then you just say OK what happens and then you look at the at the graph and you know some poverty is

affected and then you can while you can you can use it in your considerations what testimony is but it position of gesture of course I I could

elevate than that but I wanted to be sure to use all kinds of different sensors to have to be suitable to what you want to acquire where sample as a source for force the lightest and in VDTs for our displacement infrared ultrasound using as the spaceman or or general

ledger but if there's a hand here with there whatever but I so be creative and so if you want to look more into sciences and stuff you can also use sensor fusion to get even more data out of 2 different like if you have 2 sensors and his sons of fusion you can get more information out of it then you can use that if you can get from either 1 of them or both of you now the 2nd part is a

mapping about which is kind of like distinguishing about instrument for a good

instrument because you can have very easy mappings like one-to-one like no do

this it up which goes down or will you not only that whatever end of the it turns out a lot of good and music instruments are not that simple because that of course you understand recruiting that it gets boring and then use who has plan some kind of like a toy like character which you wanna avoid if you want to make it public more complex because you want people to be able to mask the instrument and should be easy the soul of many-to-many mapping for example is to use multiple input variables and you Madam to multiple synthesis parameters are

meters musician doesn't that you can try out some stuff but you wouldn't really get what's really going on because you have to change all of them at the same time to get to a result that inspires the more I call it the tennis use of instrument you the musician more 1 as but but to stress gestures instead of parameters which is like a told a different the coding of off off using instruments because the mapping ideas also you can use machine learning to

classify gesture for example you could record a gesture like like this with any sensitive detached or just using instrument record the gestures and then use a classification algorithm to to say you have to have 3 or 4 gestures and is which

1 was the closest and then you gotta connection based on that but you can also use in indirect beta and it's a you have but you have a slider and you you can of course read where it is but also you should think about how can I use the velocity of someone citing decided and between other sciences very but also you can user-defined mappings follow you should be careful of this because of its it's more flexible but the news of the face of your instrument because while it was the designed to sound like this will not be dealt with the word like this and if you make a toll random then it's random than

but you have a lot of cool

open source mapping tools I like live map awakened even map but over a whole network where using always open some control and also other interesting tool boxes from you come wake and this this is where you can do that the whole gesture of recording and and justification

and the songs and this is the said

I'm not talking so much about this of at this several different techniques attitude subject to such subjective f you have who physical modeling stuff where you actually emulator we lot instruments as digitally and you have a lot of but I great open source software to to build your own and feedback so if you

playing a violent then you but you Dubowitz and the whole body as she vibrates when he was funded recognize it but this very important for them but when they're running an instrument that they get the vibrations on the body but if you if you take that away would be way harder for them to learn instrument and it gives them a feedback about the state of the instrument the problem is not all with the 2 music instruments this doesn't occur naturally so you would have to add actuators instrument to give some kind of primary feedback to the but to the former it can that was lower by the ocean waters so was whatever can convey information to haptics visual or or even if there's a special cell that appears so yes I want

inspire the home community to to to to do it to build it is a digital music instruments of think it's a very exciting field it's complex enough so you can spend your whole life in this field and you have a lot of tools that you hand everything's getting

cheaper the Census and now we'd like those some this often the eighties there was super expensive now this is really doable but then and you have all those are embedded systems like this reply readable black etc. etc. where you can do no synthesis on this space on the critical space which was which is amazing for most people that's but to build

those other instruments that 30 years ago all the

challenges though but this by example at the Mannheim conference this only instruments but a a lot of them sound kind of the same because they have a crazy interface effort somehow and then benefits as noise this is like the and nobody can to this other than people that really like noise music and often and my eyes it's also niche is to use like yeah I do more music music so it'll have to care about the sound told don't make this a stage and and also don't be only 1 systems where this was something that I did in the past I I instruments and I tried to be good at it but actually what should have done is give it out make 2 or 3 of them and give them to musicians

and have their feedback immediately and make that's also I have to make the set of using for example an audiences like if this so as if these cable than you actually more near Mackinaw new instrument the so make it that's also like you know the because thing is to have a that an embedded system a box like an instrument but people I used to because it is very important that you know that users can talk the audience that the audience sees a like big gestures and and can relate to what the music position is doing if if they don't even see what you do on the keyboard and they won't find the forms interesting is only as not only about a music is also about the visuals but and escort that want to bring is this but so this

is this the right of talking about Myerson what the perfect game I would look like and it has a low entry fee and

what can pick it up and the media makes all that without knowing much about it but it is more sitting on the atrocities means that if anyone wants to master it has to spend 10 thousand hours to be perforated just like with the Gavilan or the piano you some some of my own creations by but on the left top is the what I

call for you miss It's rather caught on a somebody and then a joystick on the top so you can but Clarke the string and then what it is like modulate the sound in in in 2 dimensions or even circular warm and the box is a device for you press record and then use it as a tape but on the tape player and you can tell a user bury going back and forth and

then you can destroy it with effects

and destroyed every every loop you destroyed in this fight until it's noise so expect there what should do this is and so in the middle of this little thing is this is my 1st like commercial object is not being instrument is in music accessory it's so big that device least but exactly fall haptic feedback you can use it as a marginal more they

produce and is now an internal to so just a short demonstration of but of those in

the city of Reading of of the mind the you because you know that if they but with a very very different experimental and here is another 1

this so we use a little too much of the of

the

of the

the the I'm a I mean just I

and you're in the years ago and I'm a mere in

a little in this

an in

the the OK I

think analyzes the latest the

of that's also but you do we have basically we we don't have any time and we have 3 minutes late but we are on its last talked before the the the lunch break so we have some time for a fortune and if there are some questions please the other 2 microphones all situations

my when you play again May you play again your music as knowledge as in asking that happens interest I I felt like try when you my talk would last like you a little bit more than half I right and I don't wanna do this as well it I can I can set up in time century or you can come to the White religion and really tried out itself but if you want though of please do you ever tried the brain waves as a social for sound yes I should have decided that I lifted out of it there is a there was who was then there was a famous musician and developer that uses brain waves to make music but like huge helmet and stuff although I I think it was very difficult to move the because of the noisy to actually think about music and then you have and be music at the grid the still illiterate away from this stage but the people working on this yes I think even perceive as there I was thinking about something more apparently like an animal moving your the brain and late late enough in some zones so in the future that's but well with you the you can also just measure the EMG signal that comes directly from the and where somewhere was a nice project in Montreal where but there was 1 Goering hearing he grew but to you she asked Method actors to to have like the Piast and like with a syringe in the Amish live painful but to get the best signal out of the and then she asked them to act emotions and some cited those emotions into music that woman was made by Roberts so you can do it has half of the EMG signals and who thanks someone else and yeah just a very massive question more what instruments have with you like in the it's like what can you try out there but the only 1 is we set up as this 1

the article fully harmonic the other 1 if someone is interested in this is also like the whole the instruction on all the builders my website so we can actually make it work somewhere correct now that if the if but we can we can make it where generic in a couple of hours ago wrong OK as well so

carrots thank you Julian for a year and a half