Music is the Instrument

Video thumbnail (Frame 0) Video thumbnail (Frame 1170) Video thumbnail (Frame 2693) Video thumbnail (Frame 3926) Video thumbnail (Frame 5479) Video thumbnail (Frame 6285) Video thumbnail (Frame 7601) Video thumbnail (Frame 8839) Video thumbnail (Frame 9508) Video thumbnail (Frame 10482) Video thumbnail (Frame 11507) Video thumbnail (Frame 12632) Video thumbnail (Frame 13440) Video thumbnail (Frame 14903) Video thumbnail (Frame 15902) Video thumbnail (Frame 16977) Video thumbnail (Frame 18860) Video thumbnail (Frame 19811) Video thumbnail (Frame 20418) Video thumbnail (Frame 21575) Video thumbnail (Frame 22303) Video thumbnail (Frame 23376) Video thumbnail (Frame 24221) Video thumbnail (Frame 25879) Video thumbnail (Frame 27533) Video thumbnail (Frame 28832) Video thumbnail (Frame 30178) Video thumbnail (Frame 30810) Video thumbnail (Frame 31484) Video thumbnail (Frame 32088) Video thumbnail (Frame 32749) Video thumbnail (Frame 33421) Video thumbnail (Frame 34036) Video thumbnail (Frame 34708) Video thumbnail (Frame 35308) Video thumbnail (Frame 35939) Video thumbnail (Frame 36552) Video thumbnail (Frame 37150) Video thumbnail (Frame 38914) Video thumbnail (Frame 41743) Video thumbnail (Frame 43029) Video thumbnail (Frame 43878) Video thumbnail (Frame 44629) Video thumbnail (Frame 45356) Video thumbnail (Frame 46463)
Video in TIB AV-Portal: Music is the Instrument

Formal Metadata

Music is the Instrument
Innovation for Impact
Title of Series
Part Number
Number of Parts
CC Attribution - ShareAlike 3.0 Germany:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Matan's incredible journey from musician & film maker to award winning inventor started with the realization that his actions could have more 'real, positive impact on the world'. In the past 2 years he has developed ways to translate heartbeats, brainwaves and motions into music while turning everyday objects such as hats, glasses and gloves into instruments, working to empower people with special needs to express themselves in new ways, speaking and performing on stages worldwide and building interactive installations for museums & galleries. As the founder of Shift, a company which specializes in positive innovation, Matan is now laying the foundations for a new kind of lab - one which works with emerging technologies while focusing on their potential to do good in the world. During this fascinating talk Matan will not only share his story, insights and inventions, but also perform a live demo of his latest prototype, which enables anyone to play music using nothing but hand movements.
Backup Sic Hypermedia Computer animation Lecture/Conference Weight Multiplication sign Digitizing Translation (relic) Bit Musical ensemble Object (grammar)
Prototype Sic Arm Touchscreen Computer animation Demo (music) Meeting/Interview Multiplication sign Right angle Bit Musical ensemble Shift operator
Arithmetic mean Group action Meeting/Interview Video game Online help Quicksort Food energy Field (computer science) Identity management
Shift operator Focus (optics) File format Direction (geometry) Software developer Multiplication sign Projective plane Ultraviolet photoelectron spectroscopy Shift operator Event horizon Demoscene Frequency Mathematics Arithmetic mean Word Meeting/Interview Different (Kate Ryan album) Video game Self-organization Position operator Condensation
Shift operator Prototype Lecture/Conference Different (Kate Ryan album) Projective plane Video game Pattern language Musical ensemble Client (computing) Computer font
Moment (mathematics) Physicalism Musical ensemble
Game controller Group action Googol Touchscreen Computer animation Multiplication sign Bit Musical ensemble Cartesian coordinate system Arithmetic progression
Point (geometry) Prototype Touchscreen Videoconferencing Musical ensemble Disk read-and-write head Cartesian coordinate system
Trail Computer animation Sound effect Drop (liquid) Disk read-and-write head God
Type theory Prototype Wave Graph (mathematics) Computer animation Lecture/Conference Different (Kate Ryan album) Multiplication sign Sound effect Cartesian coordinate system Disk read-and-write head Signal processing
Mathematics Prototype Graph (mathematics) Computer animation Meeting/Interview Line (geometry) Musical ensemble Mereology Metropolitan area network
Googol Demo (music) Lecture/Conference Connectivity (graph theory) Video game Damping Musical ensemble Disk read-and-write head Measurement Local ring 2 (number)
Metropolitan area network Pulse (signal processing) Multiplication sign Disk read-and-write head Event horizon Rule of inference Field (computer science) Element (mathematics) Prototype Wave Programmschleife Lecture/Conference Different (Kate Ryan album) Musical ensemble
Wave Googol Meeting/Interview Video game Real-time operating system Pattern language Water vapor Musical ensemble Event horizon
Installation art Lecture/Conference Meeting/Interview Synchronization Multiplication sign Shape (magazine) Musical ensemble Measurement
Curve Shift operator Mapping Meeting/Interview Projective plane Physicalism Musical ensemble Connected space
Meeting/Interview Lecture/Conference Musical ensemble Food energy
Multiplication sign Surface Virtual machine Bit Bookmark (World Wide Web) Revision control Film editing Lecture/Conference Drum memory Musical ensemble Tunis Asynchronous Transfer Mode Social class
Sic Lecture/Conference Different (Kate Ryan album) Multiplication sign Musical ensemble Mereology Shift operator Bookmark (World Wide Web) Theory Asynchronous Transfer Mode Row (database) Cognition
Word Shift operator Lecture/Conference Meeting/Interview Videoconferencing Musical ensemble Variable (mathematics) Event horizon Theory
Prototype Software developer Meeting/Interview Multiplication sign Shift operator Arm
Meeting/Interview Multiplication sign Musical ensemble Line (geometry) Machine vision
Meeting/Interview Operator (mathematics) Projective plane Right angle Line (geometry) Power (physics)
Type theory Quantum state Software developer Meeting/Interview Planning Set (mathematics) Newton's law of universal gravitation
Web 2.0 Word Meeting/Interview Moment (mathematics) Green's function Self-organization Physical system
Metropolitan area network Joystick Programming paradigm Personal identification number Computer animation Meeting/Interview Information systems Musical ensemble Disk read-and-write head
Duality (mathematics) Meeting/Interview Multiplication sign Projective plane Content (media)
Event horizon Meeting/Interview Prototype
Arithmetic mean Meeting/Interview Musical ensemble
Metropolitan area network Computer icon Meeting/Interview ACID Normal (geometry) Musical ensemble Curve fitting
Meeting/Interview Maxima and minima Variable (mathematics)
Prototype Dot product Computer animation Lecture/Conference Speech synthesis Musical ensemble Mass Event horizon Theory Product (business) Power (physics)
Shift operator Context awareness Image resolution Multiplication sign Projective plane Online help Volume (thermodynamics) Evolute Computer font Mereology Event horizon Machine vision Computer programming Product (business) Type theory Prototype Computer animation Lecture/Conference Hacker (term) Energy level Musical ensemble Position operator
Computer animation Demo (music) Meeting/Interview
Computer animation Lecture/Conference
Computer animation Universe (mathematics) Series (mathematics)
Optical disc drive Message passing Mathematics Lecture/Conference Information systems
and so the and there's been
I'm really excited to announce Martin backup Barca that its time Otterbacher bits here with us today I'm I'm taking sides it's an answer because he is currently a reinventing body percussion you could say the upcoming rock star of digital music performance art and has developed a new way to translate movements brain and heartbeat into music turning everyday objects into instruments please give a big warm welcome to Madam back of its music as the instrument
and so on and Christians a lower body my name is not time I came here from Tel-Aviv to talk to you about what I do which is basically translating physical signals into
music show some of the prototypes inventions of made and travel bit about working with the Special Needs population to enable people with disabilities to play new ways and also at the end of the talk i'm gonna demo something a working on these days which translates arm movement into music and hopefully would have placed a music on stage today as well so when we start by really going from that talk to the very beginning my background but as you can see this is what why are some of the things about talk about so it is not like a spoiler full screen right is gonna spoiler of some of the things that I thought about but my background is actually on the music and
film I came from art and for many years that's what I did I was a screenwriter and director and I was traveling the world as a musician and I was happy but then gradually asserted feeling like something was missing the sole voice inside me kept telling me that my actions in my resources and energy could be spent doing something more meaningful something that what would influence people's life directly and help improve other people's lives and as a sort thinking about what that could be ah I reached the conclusion that 2 things were missing 1 was that our sense of impact and meaning and the other 1 was a sense of innovation of making things that new that unheard of war that are actually innovative in the field and so without knowing too much about what I was doing I decided to start a
company the but this kind of feeling of not really knowing where you are but that kind of identity crisis not knowing what you wanna be when you grow up was foreign for me because ever since I was a little kid and you want to be a filmmaker and musician and that's what he became and when I started this company called it
shifts because it was a shift for me it was a change in my direction in life and I decided to our focus on innovation for purpose meaning how to use technology how harness technology innovation for positive impact for importing causes and for the 1st year 2014 of this company that's exactly what we did we worked with all start ups we worked with for organizations non-profit was palities even a hospital in all of our projects were about merging technology and in fact it was a very very educational year where all learned a lot of new things but also sigh getting into this kind of our scene of innovation and start ups and I was fascinated especially by 1 thing and that thing was of fonts for
those of you who don't know what a hyperfine is is basically an event where people come together from different disciplines in different backgrounds usually you'll find developers designers makers and they all build things together in a very condensed period of time 24 hours 48 hours 72 hours straight nonstop and once you build these things are you come and present them in the in the back of 1 show what you've made and judges of word you some kind of prize to the best teams and this format was amazing for me I sort like experimenting with different things
they could do it a high-performance usually hackathons they have some kind of Arts Center it could be a new technology could be a theme where people build around that of theme and trying to kind of create different things for it and what I found to be my thing was music technology and this
pattern started to emerge a would come to have a font of started thing we would build some kind of music technology prototype and when and as this 1 started to accumulate my life started changing because instead of focusing on the work I was doing with clients as a it focusing on the worker was doing with music technology and the more dominant that became the more the company shift shifted for working with clients to doing in-house projects and 1 I'll I'll show you some of these prototypes inventions so so the 1st thing
and also the 4 sack upon ever
did made me realize that what I'm fascinated by is the relationship between music technology and the human body how can we translate our physical body into music into sounds in a certain making this distinction in my mind between what we could do physically to control our body like moving our hands moving ahead using our voice we could use these things musically we could control them but also there are other things in our physical body more biological things that we can't really control like a hobby or brain activity these things are musical at least for me they seem very musical but on the other hand you might be able to influence them but you can't really control them
so how do you make music with an instrument you can't really control I'm gonna show an example of each of these now the 1st 1 will be the 1st half of what went to which was for Google Glass you guys remember Google Glass it was also an experiment for Google to launch the wearable device a glass of device that would basically have a camera on a microphone in all these different sensors on it it was a bit earlier
literature and we came out to Israel there was this big hack upon for Google Glass was was the 1st time any of us has ever seen it and when I was there I noticed that these different groups started forming and they were forming around ideas that I felt were kind of awkward for example 1 group was trying to make a dating application using Google Glass they reasoned that if you're the dates and you not sure what to say what to do what you have a camera on you have a microphone you have this little screen so the Google basket tell you what to say if you've ever been on a date you know it's a horrible idea why would you go with progress to date and say what the screen tells you the saying I wouldn't take us to a date but I'm I
take it with me on stage so started thinking what could you do with Google Glass on stage and what caught my my my curiosity was that every Google Glass has the sensor Goal gyroscope that basically tracks your head movements so we knows where your head is at every given point How would you
translate those movements into music so I built the Google Glass glass beads which is basically an application that turns the Google Glass into a musical instrument I'm going to show you a little video of that very 1st on music acknowledge a prototype I made all all more than if you look at the screen behind me in this video you see these
little dots that's moving it's moving or with latency and tracking my head movements to show where my head is at any given point and with this god I can control different effects on my voice right now the effects on was most 1 this is not in the morning on the x axis because of the use of wire is 1 1 1 of it controls you can also you can also movements and also this is also my it is also my OK last drop and that's some
of the things that some of the things that are i or who or of this in our around in or a war who or rural who more rural who was worth Fusarium
or so so this was the very 1st prototype I made it 1 to Google Glass fun we actually won the dating application
affordable less and that really got my appetite going on a car I we can make all these different things and especially again I was fascinated by the relationship of the physical body but if this was controlled by my head movements on the two-loop incre Weir's control effects using head movements now came the time to try new something more biological some you can't really control and so I started working with EEG to those of you who don't know EEG is basically you are a way to measure electrical ctivity in the brain so you can try and attack different brain waves different types of brain activity using this technology and I came up with far in Israeli start-up called neurosphere the developed a new way to do signal processing for EEG to detect more data from brain waves and bring activity and if you look at this graph behind me you could see art 1 of the things that inspired me to
try and use this technology because all this red and yellow data on this graph is basically brain activity in is very chaotic but the guy who were measuring here is an experienced meditators and that part where there's nothing but are a very thick red line is when that guy went into very deep trance so he was able to consciously manifest change in his brain activity and that got me thinking this might become an instrument the this might become musical Neurofeedback Using neurofeedback to create music and the way this came about the 1st prototype was working with city would be so if you would think R is a purple educ
man so he's basically constrained to a wheelchair in is very inspiring individual before he became verbal eject used to play guitar ever since then he missed playing Languedoc he missed playing music but it couldn't really do it with any conventional instruments the EEG was a
great opportunity for something to try and express himself musically so teaming up with nearest here we created this EEG musical divides the translated Sethe's in anyone's basically brain activity into music it later that day this was a hyperfine as local Tom and later that day said he was on stage in full of a lot of people just like now and he was playing music also he was the only person I've ever met was
able to consciously make the music and go on and off his mind was so sharp and focused that he could say something like the non stop the music for 20 seconds and the music would stop me personally I played many shows that the gene I could never do that because for most of the time and for most of the people this is very hard to control the most you can do is trying relax or get very agitated so the EEG could kind of measure you're all waves and detect how wake you are but for any other use it's pretty limiting this is why I thought maybe we can take this 1 step further and make a musical packed did you of the musical had was to combine both things see the Google Glass was a wearable device but it was also ready made wearable device I didn't make it I was just using it and I was curious to see what I could do if I start from scratch so we took the designer had it was me especially for us life inspir off and we added 2 components the 1st thing was a gyroscope just like the sensor in the Google Glass that could track my head movements and the other thing is buttons you see on-screen that work a enabling anyone to basically remove themselves and create layers just like the demo I showed you but the twist reduces the insight that I also put the
EEG reader so now what you're doing loops and controlling music using hand movements are your head was controlling consciously controlling the music and your brain waves your brain activity or unconsciously creating music as well basically merging the physical and biological elements of making music using the body this prototype was made in event called music Techfest which this pretty incredible event to anyone interested in this field and it's happening at the end of this month in May in Berlin so you should probably check it out this next prototype I was also made music that has this time in London both of which are 1 music
Techfest of for wearable devices this is the prototype of a glove that measures your heart beat so just like the brain waves I could convert the pulse of a person into music and gradually as a accumulated this there's different kinds of prototypes inventions I started creating shows the rule based around the physical body making music on these shows were so are
mainly made for big stages this for
example is all Microsoft stage a big event for all Microsoft and for this show we had both the EEG brain waves and the heartbeat sensor I was playing
allegedly dada was a drummer there was a victory dupla and we were running the whole thing up through water and sent to create is geometric patterns that you see on-screen behind us in real time and another show from Google I just went on stage without any instruments and all the music was made with the body 1 person was meditating on stage and kind of affecting his brainwaves life while another girl was running on stage and increasing heartbeat alive and I was using motion to create more music and all the music was done using nothing but the body using the same technology is
a start experimenting with interactive art so this for example 1 this is all the more and this is our interactive installation a made for her to a museum which is basically like this alien cocoon shape and when you stand in front of it 8 the backs your movements your body and it gives it light sound but interesting thing about it is that if 2 people stand on both sides of this he doesn't only
the 1 of them in there is both of them so once the sink in certain moving in the same way the same time the whole thing comes alive the lights in the music climax and basically measures how sync you are with another person your movements but distrib
going I made this are interactive kind of a dance thing with curve of the new mapping and motion gestures but as it was working on this project I 3rd thinking wait a minute what about positive impact you remember how for the 1st year asserted shift and I was all about yeah let's do something that matters that's help people but a lot of these things were back to just having fun musical hats and dances 3 equal but
how we helping people how we changing their lives so if you think back about so if you would the purple agent the more I thought about this the more realized this connection between the physical body music means the most for
people of disabilities the the this is a fairly
she's are young girl from Jerusalem and she has this amazing positive energy about her even though she's both cognitively and physically impaired she can do basically nothing almost all us you can really move by she can't really speak least not you know consciously the main thing they
could do other than smile but is to wear hand and when my friend their assignment came to me and said I wanna create a musical instrument full any owner to play music that was a great challenge for me because basically all they could do was more overhead and that's
what I call 1 but machine because you can't even really fine tune movements is just moving how could you really make music using nothing but that how could you make music accessible for a person like esthetic what we did was we created a of this class this is the 1st version of the glove there were more versions later and as love has 3 different modes so the 1st mode for the glove was drum mode basically every time fairly hits a surface it creates for static sound a bit and then a snare sound the that way when she would play at time after time she could play drums like this the but but but this was very empowering because she was able to start experimenting with rhythm nothing but hand movements 1 the 1 hand moving could
enable her to play drums the 2nd mode was DJ mode where basically took Phillies favorite song which is is very uplifting positive song about the Messiah and I disagree on cut it into different
segments different bars and so instead of just passively listening to the song with adequate trigger the next part of the song in time she could choose when to bring the next part of the song in so this listening experience also became an empowering active participatory experience fulfilling and the 1st mode was an even musical I took a that his mother to my home studio and the recorded over reading to is favorite children books and then took those recordings and I cut them into individual sentences that way with adequate triggered the next part of the story whenever she wanted instead of being completely dependent on her mother she could use this glove to we the sulfur story using her mother's voice and movements inspired by what this allowed me to experiment with and what it allowed other people to feel I decided to take this 1 step further and I came up of this steering golden is accessibility
pyramid the top of the pyramid you have professional musicians these are people who have technical abilities they had equipment the no musical theory and they spend money on music and also make their money from users then you have people rural hobbyists they have all the same things but usually did only make music for money they just really spends their money on it because it's their hobby then you have people who were not musicians they're all basically thinking always should sort of Darwin was 14 0 I could never drum and all these people are obviously wrong because if they wanted to any of us could just pick up an instrument at any given point and stop playing but we tend to forget about especially these populations people might want to play have all the passion medication they might be extremely talented as well but they have these challenges preventing them from playing music cognitive challenges physical challenges making it much
more difficult to experience the gift of music and my theory was what if we connect the 2 ends of the pyramid what happens if we try and words the professional musicians and especially population and this became events called discotheque stands for disability community technology and it was the 1st event of its kind in the world dedicated to the creation of music technology for people with special needs it was produced by my company shift and also by an Israeli NGO called imagine I made a video about these events and I like to show it to you guys running right there and the to do and places that benefit the
community technology variables and for musicians especially for those
of you who made characters and
the port of musicians with 4 different challenges for the contains teams
working on 4 different different ports to create for working prototypes was flying have have
have have always wanted to kill or be
able to play 2nd or and have
the main challenge was to build the actual
extension in at the same time to
make the feel and the sound of the actual
strumming before the topic to sound like an
actual on the problem of the of
the vision a and
so will the accident since then I winter
we have tried to combine the 2 and line of money time and
microphone in a kind of
power and everything will be connected to the sole modular
the next thing that on my project here is
the 1 and the right
now it's 18 years old with combat when he was 15 months and
has the brain tumor after the operation
and cover the lines also he became a distinct and have to
land on the right place and the company
has a we of the left side of the plane
which had the right hand pointing at the left hand that only 1 thing that
we use the 1 finger has on the left hand and the a T
and continue to report by supplying him the the to
set of this type of course using
the word and hopefully it
will go the other 2 organizations you was
to 1 year is a harbinger of this is
what I meant you could do more to go
glued together wire and
due to worsening you create a system
that will send me the notes on
the web so you can write scores to
going to cover all of the other countries in
the joystick we can use to write
you this going to click and the
music that you inside head can get on the
computer and from there to the audience what mn
with and and you have
you different electronics and amino
everybody is a musician and I had a much more efficient decided the most
talented brilliant people
also kind of fun and I can tell
you that was you the the prototype of of
the most of the king at that time the at
the project content that and for me I want that kind of thing
like the focus of and they
have their needs things and can benefit from them in contact with the
people especially the holidays also look for children and also the the will
envision here to make it accessible to as many people as possible in the
room there 1st and I have
never even thought that I could play I
never even thought about it the meaning is
in the eighties and the news has been I
think of this is being
fired a lover friend
the same music
and the think music is the most
powerful medium you know in the world and when
you combine that with technology which is
an acid enabler and that's the perfect lens the
norm and I had I had the
behind this is to continent in the
and in a really hope that some of
the variables that we know and
understand the in of this and
have not been on your on
a the the the
you yes so
once again going back to the music accessibility pyramid for a 2nd this event was successful in some ways because it is by a lot of people and I went on of the store and
my my theory was basically that if you take this pyramid and you connect the dots what happens is the thing reverses what helps of person with a great challenge will definitely benefit people with a lesser challenge people who've never played people for whom it's a hobby and of course a people who all their lives are evolving on music but also in other ways I learned a lesson from discotheque because as is really going on a public speaking tour speaking about this all over the world I saw how many people were inspired and touched but I also realize how difficult it is to bring these kinds of inventions these kinds of prototypes into the mass market how difficult it is to actually make them into products that assist in power a lot of people all
over the world and the reason is money this is a niche market how could you make these inventions these prototypes into fully formed products if they're not commercially viable this got me thinking a lot about what's the next step because I thought originally I should have more of these events all over the world and I did make a lot of context of people wanting to make these events but not more thought about this more realize if we make more events and more prototypes more inventions but we never make the products that actually gets to the hands of the people who need them what are we doing and this brings us to today and I'm very excited for his stock not only because this is an amazing event and your great audience also because I want announce something of a working on Kav?i? secretly for a while and it's 2 different projects the 1st of which the bigger 1 is the evolution of the company and of the vision shifts eyes on its way to becoming an innovation lab but it's not just an Innovation Lab at innovation lab for positive impact is an Innovation Lab meant to create things that would empower and help people and solve challenges of the fortunate enough to go on these tours at MIT Media Lab and we you Innovation Lab Harvard innovation Lab X I saw all these people working on inspiring projects and I ask them questions and of the seeking the advice of a lot of people much more experienced than I am and I'm hoping that by 2017 gonna launch these labs of talking to potential partners now and dreaming about making this a reality and in the meantime I'm working on my latest project which I have with me right here and I'm going to demonstrate for you guys are you ready the red ink blot alright motive so this project is called air instrument and is the 1st project for the lab this is a prototype obviously can looks like a prototype 1st let's see the sound even works
right once again in use a volume the alright so what this does is basically this is the accumulation of what I've been talking about I again when the music that has this time in Sweden and I created is from a type for an instrument is controlled by gestures by hand movements because realized that this might be the most efficient most intuitive way for people who don't play music to play music to learn music understand music this spatial intuitive movement could be what makes music accessible in a whole new way and once we made this for type it 1 the hacker font and we got into this incubation program called music bricks which is part of the European Union and we're now finishing the Commission program and going to the next level and I'm
going to try not do little demo for you guys overly
works a secular the and right the if in the in in the in
it it the it the the in in and
thank in the the
that thank
you and we can use in
the in the in the in the in the in the in
the in the in the in the
in the in the in the middle of a series of lectures that we
think is the in the universe would pay you in a ride in the
original biological gone through
thought justice to finish off
plagiarize this is finish off the odds 1 we to this 1 simple message find the sweet spot of what you're excited about what you would do if no 1 would pay you know what you think about when you get up in the morning and try and find another drive I that meets what you think could benefit the world and what the world needs and when you find that sweet spot go for it and make a change
when it was worked on federal much pain enjoy uh you have I have here the