Media Disruption Led By The Blind
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 165 | |
| Author | ||
| License | CC Attribution 4.0 International: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor. | |
| Identifiers | 10.5446/39306 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
00:00
Musical ensembleState observerMultimediaHypermediaInfinityLecture/Conference
00:50
MultimediaMedical imaging2 (number)Right angleLecture/ConferenceJSONMeeting/InterviewComputer animation
01:56
Perspective (visual)2 (number)Perspective (visual)Lecture/ConferenceJSONXMLUML
02:33
Perspective (visual)BootingNeuroinformatikPerspective (visual)Video gameBitData conversionXMLMeeting/InterviewComputer animation
02:57
Video gameComputer scienceMultiplication signVideo gameXMLComputer animation
03:26
Interrupt <Informatik>Metropolitan area networkVideo gameCodeProjective planeComputer animationXML
03:54
Texture mappingVideo gameCodeInstallation artProjective planeHomographyLogistic distributionMultiplication signComputer animationMeeting/InterviewLecture/ConferenceXML
04:31
Level (video gaming)Musical ensembleHead-mounted displayProcess (computing)Set (mathematics)Right angleJSONComputer animation
05:04
1 (number)Slide ruleGroup actionComputer animation
05:34
SpacetimeGroup actionSet (mathematics)Lecture/Conference
06:07
Multiplication signVibrationGreatest elementReflection (mathematics)XMLComputer animationJSONLecture/Conference
07:26
Perspective (visual)Limit (category theory)Lecture/ConferenceXMLComputer animation
07:55
Different (Kate Ryan album)NoisePerspective (visual)Virtual realityVirtualizationVisualization (computer graphics)Source codeXMLComputer animation
08:30
CAN busState observerInfinityComputer animationLecture/ConferenceJSON
08:55
Dependent and independent variablesAdventure gameSlide ruleState observerInfinityProjective planeMereologySlide ruleJSONComputer animation
09:26
Limit (category theory)MereologyProjective planeRevision controlLecture/ConferenceJSONXMLComputer animation
09:56
Cone penetration testPower (physics)Revision control
10:36
Hacker (term)Virtual machineMultiplicationDependent and independent variablesComputer animation
11:24
Hacker (term)MathematicsGroup actionLecture/Conference
12:10
Local GroupInfinityState observerHacker (term)BuildingQuicksortFigurate numberMedical imagingHacker (term)Group actionForcing (mathematics)VibrationComputer animation
13:11
UsabilityMereologyGame theoryXMLComputer animation
13:39
Real numberCAN busInclusion mapMeta elementWebsiteReal numberFocus (optics)Link (knot theory)QuicksortGroup actionComputing platformEvent horizonHacker (term)Instance (computer science)Latent heatInteractive televisionComputer animation
14:39
Projective planePlug-in (computing)GeometrySoftware testingIntegrated development environmentSpacetimeGame theoryReal-time operating systemReflection (mathematics)1 (number)JSONLecture/Conference
15:31
Point (geometry)Plug-in (computing)Game theory1 (number)Lecture/ConferenceJSON
16:24
Revision controlFile formatProjective planeInteractive televisionArithmetic progressionJSONComputer animation
17:01
Visual systemVisualization (computer graphics)Projective planeLecture/ConferenceMeeting/InterviewJSONXML
17:25
Computer hardwareSystem callImmersion (album)Hacker (term)NeuroinformatikReal numberProjective planeMultiplication signSystem callHacker (term)Computer hardwareVideo gameCycle (graph theory)Interactive televisionLine (geometry)Lecture/ConferenceXML
18:16
VideoconferencingView (database)Computer fileBookmark (World Wide Web)VideoconferencingPrototypeMusical ensemblePoint (geometry)Single-precision floating-point formatGame controllerTrailJSONComputer animationLecture/Conference
20:23
Hacker (term)Context awarenessExtension (kinesiology)Online helpVideoconferencingMusical ensembleProjective planeContext awarenessNoiseHacker (term)Extension (kinesiology)VotingLecture/ConferenceComputer animation
21:13
NumberTrailDisk read-and-write headPattern recognition1 (number)SpacetimeLibrary (computing)Head-mounted displayRight angleInteractive televisionLecture/Conference
22:53
Hacker (term)Context awarenessExtension (kinesiology)Online helpMusical ensembleNoiseProduct (business)Lecture/Conference
23:20
Expert systemFeedbackElement (mathematics)NoiseBookmark (World Wide Web)Disk read-and-write headLecture/Conference
24:44
NumberYouTubeInternetworkingExistential quantificationImpulse responseSuite (music)UltrasoundInstance (computer science)Object (grammar)Touch typingVibrationGame controllerFile formatVideo gameReal numberLecture/Conference
27:30
Projective planeNumberBitInternetworkingMultiplication signGame theoryCollaborationismSigma-algebraLecture/Conference
29:26
Multiplication signMoment (mathematics)JSONComputer animation
30:37
Semiconductor memoryCartesian closed categoryMultiplication signMusical ensembleDiagram
Transcript: English(auto-generated)
00:19
And I'd like to welcome you to a talk titled Media Disruption Led by the Blind, which
00:26
is by Daniel Sabio. Daniel is originally from Puerto Rico, is now a new media artist and other things which he probably will talk about as well in Berlin. And one of the topics he will be talking about is Infinite Observer, which is the
00:41
first narrative VR experience designed from the ground up by the blind, for the blind. Hi, I'm Sabio, or the glad scientist, and yeah, there's my full, long name.
01:02
And yeah, I wanna talk to you about some of my projects, but first I want to try a little experiment with you guys. So I'd like for everyone to close their eyes. And as we all close our eyes, I would like for just 30 seconds to listen, and listen
01:25
deeply to the room around you. And as you're listening, create an image of the room around you. Create an image of how many people you think are in the room. An image of all the people who may be walking into the room right now, where they're coming
01:43
from, and just do this.
02:13
Now you can open your eyes. And as you open your eyes, imagine those 30 seconds as 30 minutes, or 30 hours, or
02:25
30 years, and then imagine this as your status quo perspective of the world. Now, imagine the thousands of years spent attending to visual persuasions, and how you'd
02:42
personally interpret everyday computing with this newly gained perspective. Now hold on to that for a little bit, because we're gonna go on a tangent. So real quick, I just want to give you a long story of my life. It's basically like three conversations. So permission is granted, please, you know, if you feel like you need to interrupt, you
03:02
can do that. So when I was five years old, I told my mom I wanted to make video games. She was like, okay, whatever. I was just super into Duck Hunt. And you know, 10 years later, she's like, what do you want to do in life? You know, it's that time. I said, I want to make video games. So that led me on a crazy path.
03:21
And I ended up going to Georgia Tech studying computer science. And then I ended up in Barcelona, met this guy who changed my life, who I literally saw in the hall earlier. And I said, hey, man, you changed my life. And he was like, yeah, okay. But basically, what he taught me was that I could make art with code.
03:40
And that literally just blew my mind. And I realized that everything around me was code. And that everything around me was, you know, manipulatable. And everything could be an art project or an art anything, whatever you want to call it. And it was like this crazy achievement became unlocked in my life. So I spent about eight years after that.
04:01
Didn't really even care about video games. And just wanted to make stuff with code, installation art, VJ, projection mapping, blah, blah, blah. All that stuff. And then, you know, we're talking about VR, though. So then I realized one day, oh, yeah, there's this immersive art that I'm making all the
04:21
time. I'm making this crazy projection mapping stuff. There's all these logistics involved. What if I just make a VR experience and put people into this? Wow. And it hit me so late that it also blew my mind. And then I'm also a musician. So when I got the VR headset, I wondered if there was a way to make music in VR. So I figured that out.
04:43
And I laughed a lot while I was doing it. And I've performed on stage a ton doing this. But that's not really what I'm talking about. As I was doing that, people would come up to me after every show and say, hey, you make music for VR, right? I'm like, no, I make music in VR. But I realized that, hey, I had to learn a new skill set, because people are going to
05:01
continually ask me about this. So I quit my job. I moved to Berlin. There's a bunch of stuff about how to make stuff for sound in XR technology. If you're interested, slides will be up, all that good stuff. And I moved to Berlin. And I took a course with someone who's actually sitting in the audience and learned all these
05:21
different technologies about producing sound in a spatial audio system. And I had this question of how does a group audio-driven VR experience look. So I created an experience with a group of amazing people where we had four people blindfolded. And they're in the Monom space, which is in Berlin.
05:41
And it's a 4D sound center. So there's spatial audio. And you can walk up to sounds. And they exist in the space around you. And you can really interact with sound in a different way. And it was a good opportunity to take people into a sound-focused and tactile-focused VR world. So I did that in that setting. But the issue with that setting is that it lives there.
06:02
It lives in Monom. And you can't take that space anywhere else. But what happened during that, and lots of gratitude to everybody that I worked with, is that I realized something that I just kept on thinking about what it feels like to be blind,
06:23
and how I could never really fully empathize with that experience. And so I want us to close our eyes again. I want us to just try one more time. But this time, as you close your eyes, I want you to just feel.
06:41
I want you to feel your fingertips, wherever they're laying. I want you to feel the bottom of your elbows, the insides of your knees, the bottom of your feet. And I want you to see if you can detect vibrations around you, and see if you can feel the actual reflections of those vibrations in the room.
07:34
Now open your eyes. And you can imagine the same things again, and again, and again.
07:41
That these are minutes, and these are hours, and these are years. But at the end of it all, what we come to is that that's our perspective from our limited experience in this, and that we need a deeper perspective. So people often tell me what VR is.
08:02
People in the VR community try to shove things in different places, say that VR has to be this, VR has to be that. And well, honestly, I'm kind of like, eff that noise. There's a lot of different things, and we're creating our own paths in this world. So it's not really virtual reality, or visual reality, it's virtual reality.
08:22
So if you think you need a headset necessarily to create a virtual reality experience, maybe we could think about it in a different way. And I came back to this question again, and again, and what I realized is that it transformed into what would a VR experience be like for the blind.
08:44
And can this experience actually challenge visual culture? And that led to the creation of Infinite Observer. So, Infinite Observer is a VR experience designed for the blind.
09:02
And as I started this project, I thought, OK, I'll make a VR experience that's for the blind, but also can be experienced by the sighted as well. And, you know, people in the VR community are just sitting there calling me crazy non-stop. And I'm like, OK. So now we're going to get to a lot of slides with just text,
09:22
because I actually feel like I want to embody more so a less visual tone for this part. So there's a lot of research to do. As soon as I really seriously decided to pursue this project, I went to... So there was that realization.
09:41
I went to actually live with a blind couple in the Netherlands, Yanni and Tina Visser. And they run a foundation called the Walk-In Foundation, and they actually bring people into a dark experience, which we could say is a multi-sensory, enhanced version of the glimpse that you guys participated in during this presentation.
10:03
And so they have this multi-room experience that you go into. There's literally no light. Your eyes try to search for it for the first few minutes, and then they literally just get tired of firing the cones and rods or whatever that is. And then you give in, and you start to use your other senses, and you start to discover all of the powers that you have beyond just sight.
10:25
And so I spent about five days living with them, just night and day, really learning so much. And what I ended up learning from them was, first of all, these people are incredible. Yanni and Tina were both fire walkers teaching people how to walk on coals.
10:45
Tina was teaching people ancient songs for years, and Yanni actually was a hacker. He was building his own ham radios and has built multiple Linux machines and also has his own distillery in his house, which his liquor is really good.
11:01
And as they shared their experience and their blind experience with me, I also shared with them a ton of VR experiences, which I thought, okay, I'm going to take the best audio VR experiences, I'm going to show them to them, they're going to have a response, they're going to say, yeah, this is good, yeah, this is bad. Honestly, there's zero UX accessibility for people who are blind in VR at all.
11:26
There's nothing to do with voice menus, there's no thought whatsoever. And yeah, it made me pretty sad at first, but I also realized that there's a great opportunity to make a change.
11:42
And I think that as I was with them, I really got inspired for the narrative of the piece as well, because of the background, and as we were talking, they were saying, well, what if we had this group of underground hackers who are interested in overthrowing the dominant visual culture?
12:04
And that seemed to resonate a lot. So yeah, there's a little reminder. So that really sparked the story building. And this is, oh, you can barely see it, cool, that's fine.
12:21
So this is the basic image that sort of led our guidance in that it's smoky, there's an unknown figure, and you're welcomed into this group of hackers who want to subvert these dominant forces of society, and they want to use vibrational forces to do so.
12:41
So instead of typical ways, they may use echolocation to navigate. They then use molecular vibration to actually teleport. And then they use, by mapping the tectonic shifting because they're actually underground as well, they're able to detect movement above and also detect things
13:05
which sound can actually disrupt visual vibrations in the visual spectrum. So these are some of the things that we talked about and actually became parts of the game, and we're able to implement already some of these into the experience.
13:24
So as I mentioned, echolocation, we also have a frequency matching gun where you're able to hold it up to different materials and match a frequency, and that's able to explode, open up new pathways in this underground world.
13:41
And really what the narrative is about is kind of telling a meta-narrative and how can this actually include a real society of blind hackers, potentially, through just this experience and this art. So how can we take a site-specific installation, for instance,
14:03
that takes place at a specific festival or at a specific community event, Focus on the Blind, and have this merging of, okay, at the end of the experience, someone in the narrative is inviting you to join this group, and as you leave the experience, maybe you have a text on your phone with a link to a website
14:24
that now has access to this community. These sorts of interactions, and then how is it feasible to create an audio-only group community platform that would be easily navigable by the blind. So these are a lot of the things
14:41
that we're thinking about in this project now. So the technical side of things is pretty interesting. So I teamed up with a audio designer who specifically specializes in spatial audio, and he works at Magic Leap, and with him, we've been building out the audio environment.
15:01
And so we're using Wwise, which is a popular spatial audio sound engine for Unity and Unreal. And because of the plugin that they have, it's able to actually detect the real-time game geometries and create realistic reflections of the sound in the space. So as we test this, we're looking very strainously
15:23
at how realistic this sounds for the blind. Like, is this passing the test? And so far, it's been questioned, and we're wondering, is technology not there yet? Is this something where we have to continue
15:40
deeply researching more and more and maybe use some cutting-edge research institutions to reach that point? And, yeah, as I mentioned, we're working in Unity, and originally, we were focused on deploying it to headsets. But I think that with the small enough market
16:02
of the headsets that blind users actually have access to, it makes more sense now to look at both PC users as well as mobile users, and in using some of the Wyze plugins in a game engine setting, it's easy to be able to deploy it to both
16:22
and be flexible in that way. So those are a lot of the different things that we're considering. And can an audio version only exist? Can we create custom solutions that can travel
16:41
that maybe combine all of these into installation formats as well? These are all different things. If you haven't noticed, this is an in-progress project. And so this is the team. That's Yanni and Tina, and that's Kedar, and this is Chris, who's been guiding a lot of the interaction and creative along with me.
17:05
And really, the goal of this project is to empower the blind and create a heightened appreciation for our senses beyond sight. And through the narrative, hopefully it will instill some clarity around the dominance of visual culture and the ways that all of these tech giants
17:22
have been using visual culture to greatly manipulate people, and it may inspire some new ways to circumvent this. And in a bigger sense, I would love to create a voice for the blind as we develop new technologies. I really do see immersive technologies
17:40
as being the next big way that we interact with computers for the next 10 years, and I would love to see this be potentially a wake-up call or, along with a slew of other projects, be a wake-up call that this needs to be done ahead of time of projects coming out, or, sorry, hardware and these cycles coming out.
18:03
And then, yeah, creating a community for interaction between real-life blind hackers to be able to discuss anonymously and even maybe practice some of these techniques together that are inspired by the narrative. Yeah, and I do have a video,
18:21
so I'm gonna show that now. So this is a video of an early prototype. You usually find us, track us through the echoes,
18:42
but you, you're the breaker.
19:01
I know, you're wondering why we're all here, so let me break it down for you. We have never had a voice. We're always just an afterthought, and they think we are powerless, and until now, we believe them. That's not a problem.
19:21
That is our advantage. They've made us invisible, so they'll never see us coming. We'll start with the US. By finding their single point of failure, we can break down the greater infrastructure. Look, the US controls the entertainment market. If we are able to infiltrate the system
19:40
and corrupt it without detection and insert our beacons, we can start to find others like us and accelerate our bigger initiatives across the world.
20:35
Oh, that was a video. So yeah, I'm happy to open it up to questions now,
20:42
and I actually have a few questions for you guys because I know that probably, I'll just say all of you are smarter than me, so I'm very interested to know if anyone has experience using sound or noise in the context of hacking and what people think of the community extension
21:02
of the piece, and if there's research out there that could possibly help this project that I'm not aware of. If you have a question, please walk to one of the microphones. Microphone number two.
21:21
Hi, have you given any thought to the fact that currently, as far as I understand, you need a big installation, right? So you cannot have this experience at home where if you're a visual, you can just buy the headset and that's it. Have you given any thought into building a audio VR headset?
21:44
Yeah, so this is something that I was trying to actually talk to Oculus about doing and basically extrapolating the sensors that they have into using it for head tracking because when you're in an ambisonic world, the main thing that lets you interpret it as 3D audio is the sensors being able
22:04
to know when you're turning your head or when you're moving within that space. They kind of just brushed me off and didn't give it much thought. But it is something that I'm considering and also that's kind of why I mentioned looking towards some AR focused libraries
22:24
where it has the same spatial audio features and you're actually able to scan a room with your phone and that may just be a more readily accessible to where, okay, someone's wearing headphones and they're able to scan their room with their phone and then get audio cues and also have voice recognition where they could guide their character
22:42
through the space as well, which is something all of these people are super familiar with, is talking to their phone and having those interactions. Thank you. Okay, microphone number one. So thank you very much for doing this and for the talk, it was really interesting.
23:02
I have some experience of listening to 3D noise, 5D music, they call it as well, which is easily done with Ableton in a normal headset and my question to you would be, since I think it's not developed yet, you don't have a final product,
23:21
if you ever thought of doing something that actually blind people could learn how to listen the reality in a better way, increasing some ways to pay attention to some kind of noise or feedback that would only require a normal headset
23:41
because closing my eyes once and listening to the sounds of the guy shaving your head, like the VR experience of that, it's quite amazing what you can do with normal headsets, you know? Yeah, that's a really interesting idea to have like an educational element in there.
24:06
Yeah, that's super cool and I love that you brought up the binaural barbershop example because I actually showed that to Yanni and Tina and it was one of their favorites, you know, beyond all the VR and ambisonic stuff there, like that one, it was crazy.
24:22
So binaural definitely has its place and even talking with like Kedar, who's like this expert in spatial audio, he's like, I don't, they've been trying to explain it but there's some psychological phenomenon with the way that binaural works with our brains and it just kind of psychologically fakes us out in a cool way, yeah, so thank you.
24:44
Thank you and if you never watch, watch the barbershop for audio experience on YouTube. Again, microphone number one. What kind of equipment have you used for haptics or have you played around with haptics?
25:01
Yeah, so all that we've used so far is just the basic Oculus Touch controllers. I've applied to a couple of grants with haptics research teams to look into that more. We were going to talk to Hardlight and then they went under. What's Hardlight? Hardlight is like, it's like a full suit for haptics
25:25
which also has its pluses and minuses, especially in an installation setting like switching over between people that can be like pretty crazy but something that I'm looking into is something more simple, that's like a wristband that creates basic vibrations
25:42
that actually like send impulses to all of your hands so you feel like you're feeling things and then the ultrasound stuff looks super cool. Do you have experience working with haptics? Yeah, I played around with like haptraators. They basically create an illusion on your skin that you feel like something's,
26:02
like for instance, dragging you through the air. There's a psychologist who's working with visually impaired people called Vincent Highward. He has a lot of really crazy experiments. Basically, it's like a vibrating motor that it's hard to explain, you have to feel it
26:22
but if you hold that thing, it just like kind of drags you through the air and you can do a lot of really crazy stuff with it. But yeah, just Google Vincent Highward. He's a crazy guy. Awesome, thank you. Before we shoot some microphone number two, there are questions from the internet. Oh, cool. Take my angel.
26:51
Yeah, definitely. That's why ideally, this is an installation based experience where you are able to integrate
27:01
even real life objects as haptics and also scent, possibly taste, which is something that I did explore with the blindfolded experience that I'd love to bring into the installation format. But yeah, a lot of it is technology becoming more accessible, like with the haptics.
27:21
And the more that those devices become more prevalent, the more that they can kind of enter into the work. Microphone number two. Hey, thanks a lot. A really inspiring project. And I would really love to hear a bit more about the collaboration you have
27:41
with the three other people. How do you work together? Are you in one place? And can you share a bit more of these experiences? Yeah, definitely. So we had the initial idea, which was kind of in my brain. And then I emptied my brain to Yanni and Tina. And we really started to go crazy
28:02
on the conceptual side of stuff. And I talked with Kedar about it, who had made a VR experience for the blind a few years ago. And then we started developing. He lives in San Francisco. And so we would work over the internet together.
28:21
And then Chris, I knew from living in Atlanta and pulled him onto the project. And it's kind of been me traveling to all these different places for really intense hackathons. And then us working remotely as we can. The next question again for microphone number two.
28:43
Because you asked for research that could help you with your project. You probably know, but just to be sure, there is an existing community of audio games and people who make those. And there's a researcher in Vienna who's called Michael Urbanek,
29:00
who does research on them. That might be interesting for you. Cool. Yeah, I'll definitely look him up. Thank you. Okay, any other questions? Sigma Angel, do we have new questions from the internet? No questions. Do you have new questions for the audience? I...
29:22
I just wanna actually get us to do this one more time. If you guys are up for it. So this time as we close our eyes, I want us to really take a moment to imagine ourselves having been this way before,
29:44
as we have experienced this before. And I want us to use feeling and listening and scent and taste and to try to create an imaginary world around us and to have that feel as real as possible.
30:37
Thank you so much for your time and all your everything
30:42
and enjoy all of this.
Recommendations
Series of 19 media