Augmented Reality: Bridging the gap between the physical and the digital world
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 165 | |
| Autor | ||
| Lizenz | CC-Namensnennung 4.0 International: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen. | |
| Identifikatoren | 10.5446/39223 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
00:00
HalbleiterspeicherFormation <Mathematik>PrimidealRelativitätstheorieComputeranimationVorlesung/Konferenz
00:37
BeobachtungsstudieRelativitätstheorieDatenfeldInteraktives FernsehenGüte der AnpassungBitErweiterte Realität <Informatik>DifferenteVorlesung/Konferenz
01:41
Erweiterte Realität <Informatik>Materialisation <Physik>Erweiterte Realität <Informatik>BitTermAntwortfunktionVirtuelle RealitätComputerspielVorlesung/KonferenzComputeranimation
02:24
TermVirtuelle RealitätNeuroinformatikKartesische KoordinatenSchnittmengeBitZweiRechter Winkel
03:09
Erweiterte Realität <Informatik>Inhalt <Mathematik>MultiplikationsoperatorVirtuelle RealitätKontrast <Statistik>DifferenteErweiterte Realität <Informatik>TabelleKartesische KoordinatenObjekt <Kategorie>CyberspaceEin-AusgabeComputeranimation
04:10
Erweiterte Realität <Informatik>Virtuelle RealitätCyberspaceTabelleMixed RealityMereologieVirtuelle RealitätErweiterte Realität <Informatik>Kategorie <Mathematik>KontinuumshypotheseKlassische PhysikMaßerweiterungComputeranimationVorlesung/Konferenz
04:48
Mixed RealityVirtuelle RealitätErweiterte Realität <Informatik>Erweiterte Realität <Informatik>VererbungshierarchieInhalt <Mathematik>BildschirmmaskeImmersion <Topologie>GruppenoperationBildgebendes VerfahrenMultiplikationsoperatorSmartphoneRechter WinkelProgrammierumgebungMobiles InternetBeamerComputeranimationVorlesung/Konferenz
06:15
Erweiterte Realität <Informatik>SpieltheorieRotationsflächeTouchscreenDatensichtgerätErweiterte Realität <Informatik>SpieltheorieRotationsflächeDatensichtgerätTouchscreenBildschirmmaskeTermPerspektiveComputeranimationVorlesung/Konferenz
07:08
Erweiterte Realität <Informatik>MathematikVirtuelle RealitätReelle ZahlLeistung <Physik>RelativitätstheorieURLComputerspielDatensichtgerätPhysikalismusObjekt <Kategorie>EinsSmartphoneComputeranimation
07:45
SmartphoneIntegralMereologieProdukt <Mathematik>ComputerspielFokalpunktErweiterte Realität <Informatik>MultiplikationsoperatorVorlesung/Konferenz
08:24
Quick-SortMultiplikationsoperatorErweiterte Realität <Informatik>Virtuelle RealitätTechnische OptikFrequenzKlasse <Mathematik>TermWeg <Topologie>
09:28
Virtuelle MaschineMultiplikationsoperatorMobiles InternetBildschirmmaskeWellenpaketKartesische KoordinatenTuring-MaschineBeweistheorieNotepad-ComputerErweiterte Realität <Informatik>CASE <Informatik>Weg <Topologie>Generator <Informatik>Vorlesung/Konferenz
10:36
Erweiterte Realität <Informatik>Vorzeichen <Mathematik>Mixed RealityRenderingAutorensystemDatenmodellSteuerwerkGenerator <Informatik>Erweiterte Realität <Informatik>FlächeninhaltKartesische KoordinatenBitRenderingCASE <Informatik>ComputerspielStatistikPunktMixed RealityLogistische VerteilungGruppenoperationComputeranimation
11:36
Lokales MinimumTelepräsenzTelekommunikationBitElektronischer ProgrammführerCASE <Informatik>SprachsyntheseAvatar <Informatik>ProgrammierumgebungComputeranimation
12:20
Bildgebendes VerfahrenRechter WinkelPerspektiveUmsetzung <Informatik>Zentrische StreckungReelle ZahlTabelleVirtuelle RealitätComputeranimation
13:05
Physikalisches SystemSmartphoneGeradeVorlesung/Konferenz
13:58
DifferenteQuaderAbstraktionsebeneGüte der AnpassungSmartphoneMAPGebäude <Mathematik>MenütechnikPunktInformationMatchingTranslation <Mathematik>Formale SpracheVorlesung/Konferenz
14:45
GoogolFormale SpracheInhalt <Mathematik>Translation <Mathematik>LoginZeiger <Informatik>AbstandSchreib-Lese-KopfKontextbezogenes SystemRichtungInformationObjekt <Kategorie>Vorlesung/Konferenz
15:39
CASE <Informatik>Kartesische KoordinatenCyberspaceWeg <Topologie>Virtuelle RealitätDatenfeldErweiterte Realität <Informatik>
16:39
Leistung <Physik>ExpertensystemPhysikalisches SystemAggregatzustandInformationVorlesung/KonferenzBesprechung/Interview
17:17
InternetworkingBenutzeroberflächeInformationRouterErweiterte Realität <Informatik>Peripheres GerätVorlesung/Konferenz
18:09
InformationGraphMultiplikationsoperatorDrahtloses lokales NetzDirekte numerische SimulationServerBandmatrixEinfach zusammenhängender RaumFehlermeldungWeb SiteRouterZeitrichtungVorlesung/Konferenz
19:13
BitWeg <Topologie>MultiplikationsoperatorOrtsoperatorAdditionAggregatzustandProgrammierumgebungComputeranimation
20:05
MultiplikationsoperatorProgrammierumgebungEndliche ModelltheorieAbstandPolstelleAntwortfunktionStrömungsrichtungWeg <Topologie>DatensichtgerätDifferenteInhalt <Mathematik>RenderingSchreib-Lese-KopfVorlesung/KonferenzBesprechung/Interview
20:59
Message-PassingDatensatzDatensichtgerätTeilmengeInhalt <Mathematik>BeamerObjekt <Kategorie>Vorlesung/Konferenz
21:39
DatensichtgerätObjekt <Kategorie>AggregatzustandTechnische OptikMinimalgradDatenfeldSichtenkonzeptKonditionszahlBesprechung/InterviewDiagramm
22:48
AusgleichsrechnungProzess <Informatik>Schreib-Lese-KopfZustandsmaschineObjekt <Kategorie>BildschirmmaskeÄhnlichkeitsgeometrieDienst <Informatik>Web ServicesDatenstromHead-mounted DisplayPunktKonstanteVisualisierungEin-AusgabeEinfach zusammenhängender RaumStreaming <Kommunikationstechnik>InformationService providerQuellcodeInformation RetrievalVorlesung/Konferenz
23:59
BildschirmsymbolKreisbewegungBildgebendes VerfahrenServerObjekt <Kategorie>Information RetrievalComputerspielHead-mounted DisplayDatensichtgerätMailing-ListeCASE <Informatik>KonstanteBereichsschätzungDienst <Informatik>
24:59
Objekt <Kategorie>Rechter WinkelMereologieNormalvektorStrömungsrichtungPunktFormfaktorPaarvergleichVorlesung/Konferenz
25:37
MereologieInhalt <Mathematik>Interaktives FernsehenObjekt <Kategorie>UltraschallInterface <Schaltung>ProgrammierumgebungAggregatzustand
26:28
SprachsyntheseGamecontrollerTouchscreenSmartphoneInteraktives FernsehenProgrammierumgebungStörungstheorieWasserdampftafelInhalt <Mathematik>Rechter WinkelEin-AusgabeVorlesung/Konferenz
27:10
Reelle ZahlErweiterte Realität <Informatik>Inhalt <Mathematik>InformationsüberlastungGamecontrollerInformationArithmetisches MittelInformationsverarbeitungLastDienst <Informatik>ComputeranimationBesprechung/Interview
27:48
InverseInhalt <Mathematik>Automatische DifferentiationComputersicherheitsinc-FunktionDatenmissbrauchStrömungsrichtungStreaming <Kommunikationstechnik>OrtsoperatorDienst <Informatik>DifferenteSystemplattformHardwareComputeranimationVorlesung/KonferenzZeichnung
28:48
SystemplattformStrömungsrichtungGamecontrollerRichtungSmartphoneSensitivitätsanalyseSoftwareMultifunktionNichtlinearer OperatorXMLComputeranimation
29:29
FlächeninhaltSmartphoneKartesische KoordinatenErweiterte Realität <Informatik>Schreib-Lese-KopfDatensichtgerätPhysikalisches SystemKontextbezogenes SystemDesign by ContractMobiles InternetVorlesung/Konferenz
30:26
Kontextbezogenes SystemErweiterte Realität <Informatik>RuhmasseMultiplikationsoperator
31:08
Spezielle unitäre GruppeMathematikKonstanteErweiterte Realität <Informatik>Vorlesung/Konferenz
31:46
ComputerspielURLMathematikInformationsverarbeitungCASE <Informatik>GeradeMaßerweiterungVorlesung/Konferenz
32:28
Elektronisches ForumErweiterte Realität <Informatik>Objekt <Kategorie>Wort <Informatik>Vorlesung/Konferenz
33:19
Erweiterte Realität <Informatik>ComputerspielFormation <Mathematik>Güte der AnpassungMathematikRechenwerkSchnelltasteBitZahlenbereichRechenschieberOrdnung <Mathematik>Visuelles SystemPortscannerRechter WinkelVorlesung/Konferenz
35:29
Rechter WinkelMehrrechnersystemInformationBeamerBildgebendes VerfahrenErweiterte Realität <Informatik>Reelle ZahlCyberspaceStreaming <Kommunikationstechnik>InternetworkingMereologieDatenfeldRuhmasseTouchscreenInverser LimesVersionsverwaltungSchreib-Lese-KopfWeg <Topologie>BitHardwareEin-AusgabeRechenzentrumVisualisierungBildverstehenServerProzess <Informatik>DatensichtgerätVorlesung/KonferenzBesprechung/Interview
38:32
Erweiterte Realität <Informatik>InternetworkingMAPTeilmengeSmartphoneFunktionalRichtungVorzeichen <Mathematik>Anpassung <Mathematik>Güte der AnpassungTranslation <Mathematik>CASE <Informatik>ATMFormale SpracheVorlesung/Konferenz
39:43
ZahlenbereichErweiterte Realität <Informatik>Virtuelle RealitätRechter WinkelFacebookSystemplattformRichtungVorlesung/KonferenzComputeranimation
40:26
ComputerspielDatenfeldBildverstehenGenerator <Informatik>InternetworkingSmartphoneDatenmissbrauchMathematikInformationErweiterte Realität <Informatik>Vorlesung/Konferenz
41:27
ZahlenbereichDatentransferErweiterte Realität <Informatik>Leistung <Physik>SmartphoneServerNeuroinformatikÄquivalenzklasseEndliche ModelltheorieProzess <Informatik>BildverstehenPunktwolkeStreaming <Kommunikationstechnik>Mooresches GesetzProdukt <Mathematik>Physikalisches SystemNormalvektorOffene MengeStellenringDatenmissbrauchTaskGebäude <Mathematik>Mixed RealityVisualisierungKonstanteHardwareKomplex <Algebra>Projektive EbeneURLPunktInformationMultiplikationsoperatorProgrammierumgebungMereologiePrototypingRechter WinkelVirtuelle RealitätCASE <Informatik>Güte der AnpassungSoftwareEin-AusgabeÄußere Algebra eines ModulsInterface <Schaltung>ComputerspielOpen SourceVorlesung/KonferenzComputeranimation
46:26
Erweiterte Realität <Informatik>BeweistheorieObjekt <Kategorie>Information RetrievalResultanteFormation <Mathematik>CASE <Informatik>MAPDatenbankDienst <Informatik>InternetworkingBildgebendes VerfahrenOpen SourcePunktVorlesung/Konferenz
Transkript: Englisch(automatisch erzeugt)
00:18
We're going to look at how to bridge the gap between the physical and the digital world.
00:24
So we go into immersive things. And my guest here, our guest here, is Patrick and he goes as well with the nickname Prime. He's currently doing a PhD on human-computer relations interaction.
00:44
It's an important difference, I would say. Okay. I'm not going to discuss this since I'm as well a little bit on this field. We made an agreement about that. But he's focusing on augmented reality today.
01:01
He's doing this study and Dresden I understood. He has a general interest, of course, in things to do with 3D that goes along with these kinds of things. And his talk provides us an overview on AR in general and explains its possible uses for goods and for the bad.
01:27
I won't say so. Okay. Ladies and gentlemen, here is Patrick. Please give him a warm welcome. Thank you. Light the fuse and put it in play, Patrick.
01:41
Thanks. Yeah, thanks. Hi, everyone. So this is a foundation talk. So it's mostly aimed at people who know nothing or very little about augmented reality. So hopefully after the talk you will know a little bit more. However, if you have some experience, I hope maybe you will get some new impulses. I don't know.
02:02
Furthermore, this talk is also focused more about how augmented reality can be used for everyday life and not so much on the scientific community. And it's roughly like 30 minutes or something like that. So I made it a little shorter so that we can have some questions in the end. So if you have something that's of particular interest, just ask afterwards.
02:23
So at first, let's talk a little bit about virtual reality, because like I think most people maybe are more familiar with the term. There's been a lot of talk about virtual reality, like changing and revolutionizing the way we use computers. There are as many commercial virtual reality headsets as ever before and also quite a lot of applications.
02:46
But somehow VR doesn't really influence our everyday lives that much. I think there are two reasons for that. The first one is that virtual reality applications are mostly centered on entertainment right now.
03:01
And the second is that it disconnects us pretty much from the real world. So while it might have the ability to actually transfer us into another world, which more or less looks like this, at the same time we are caught up from reality. And in contrast, augmented reality expands the real world.
03:21
And in the mind of most people right now, this looks something like that. But the important thing is that we can still perceive our surroundings. And we can also experience the virtual content as well. And this particular difference makes AR a lot more useful for everyday use than virtual reality.
03:40
So let's go to a little definition of what augmented reality actually means. So the important thing is that we combine real and virtual content, blowing the borders between both. So as you can see, here's a virtual globe on a real table. And the application also interacts in real time.
04:01
So if I move or if I have some input, the application reacts on the spot, there's no delay. And the objects are situated not relative to the user, but in space. So if I move around, the globe stays on the table. So this is basically, when I'm talking about augmented reality, what I mean.
04:22
But it's also important to know that AR is only part of the so-called mixed reality continuum, which ranges from reality without any augmentations. So the normal reality we experience every day to a completely virtual environment.
04:41
And those four categories are basically the classic mixed reality continuum. There's also some extensions to that, which you can see here. I don't want to go into any detail of what those all mean, but the important thing to remember is that there is no hard border where augmented reality begins and where it ends.
05:04
There's also a lot of ways to provide augmented reality. Mobile augmented reality, like Pokemon Go, where the AR content is perceived to a mobile device, like a smartphone, is the most common form today. There's also projected augmented reality, where you have a projector superimpose an image onto a real environment,
05:26
which is especially useful if you have collaborative scenarios where the content should be perceived and manipulated by a group of people at the same time. You can actually see this example right out of the door.
05:42
There's a similar installation there. And then there's what this talk is also mostly focused about. Head-mounted augmented reality, where users wear AR glasses, which can be seen through, and where the virtual content is projected onto. And this is the most immersive form of augmented reality.
06:04
And when you have sophisticated technology, it's more or less indistinguishable from the real world. So that's basically the goal of what we want to get. So, but now it's the question, okay, why augmented reality? What's so important? Why have this talk at all?
06:22
Yeah, for that, I want to quote Tim Sweeney, the CEO of Epic Games, because it's always a good idea to quote someone who's more famous than oneself. And he said that he believes that augmented reality will be the biggest technological revolution in our lifetime, because once you have an AR display, you don't need any other form of display.
06:43
You just take your screen wherever you go. So that's a very powerful thing to say. He also thinks that the revolution will happen in the next 10 years, which is very optimistic from my perspective, when you look at what the current technology can do. But nonetheless, I believe this revolution is going to happen sooner or later.
07:03
So it's useful to familiarize yourself with the term. What is also important to note is that the goal is not to just substitute a real displays with virtual ones, but augmented physical objects themselves that the augmentations really have a relation with the location they are situated in.
07:24
So only then AR will have really the power to revolutionize our everyday life. So I will elaborate on that a little later on. But first the question like how likely are fundamental changes in our society triggered by augmented reality?
07:42
So if we compare it with smartphones and the way they are used, I think it's very likely. Because smartphones today are technology focused directly on supporting the way people live their lives every day. So it also started as a research a long time ago before commercial products were available.
08:01
And now they are ubiquitous, they are an integral part of people's life. And for some people it might be difficult to actually live without them. And I think when AR is likely to do the same when it's focused on the things people actually need and once the technology comes along.
08:21
But before we go there, a short, very short history of augmented reality so that you can see where this is all coming from. So actually augmented reality is rather old. So the first augmented reality virtual reality device was built by Ivan Sutterland in 1968.
08:40
It was called sort of thermo class. But it had already had tracking and see-through optics. So that's quite advanced for the time I would say. The actual term augmented reality was only coined 25 years later by Kaudel and Mitzel in 1992.
09:01
So quite a long time in between. But of course the research stood not still during that period. Here's an interesting example that I wanted to show you from 1994 by Stade et al. Which shows an ultrasonic image of a human fetus directly superimposed over a woman's belly.
09:23
And it shows that even very early on that people were thinking about how we can use augmented reality to really benefit humans in a lot of ways. Despite the technology at that time being very limited. This is also important step because it was the first mobile AR headset for usage outside of the lab.
09:44
With the wonderful name Turing Machine by Feiner et al. in 1997. So even early on the question was how can we use this technology not only indoors but how can we use it outdoors as well. Which is very important.
10:02
And then finally the first AR application using mobile devices. In this case PDAs for the people who still remember them with extended cameras. And you can see those little markers in the background which are tracked by the cameras. And then little virtual trains are overlaid over the wooden train tracks.
10:23
So it's not really that practical just to like a proof of concept more or less. So but yeah like I said mobile AR today is the most common form of augmented reality. But maybe that might change soon because now we have like the first generation of really commercially available augmented reality headsets.
10:46
And therefore also an increased interest from the scientific community but also from the industry for solving their various problems. So this immediately brings us to the areas of application and hopefully how augmented reality may influence our lives in a positive way.
11:04
First a little bit about the research. Actually only 12.5% of the research is actually focused on application cases for augmented reality. The statistics are based on the international symposium on augmented reality.
11:22
And the rest is focused on things like tracking technology, interaction, rendering and so on. And of those application cases most of them are usually industry centered and not so much on everyday life. So we have things like for example logistics, we have things in the medical sectors.
11:41
And of course they have a great indirect influence on our lives. We will still benefit from those things. But I want to focus a little bit more on everyday use. So how can we immediately benefit from augmented reality and explore that a little further. Those are things like guides and cultural heritage for example and telecommunication and telepresence.
12:03
Which also brings us to the first use case which is AR telepresence. Which is more or less like video phoning only more with 3D avatars of the person you are speaking with embedded directly into the environment. So basically like a virtual avatar sitting on the chair next to you.
12:22
Which is I would say rather cool because it's something that's out of sci-fi novels or films but now it's here. Now we can use it and despite being far from perfect it's still very incredible. At least from my perspective. And you can not only have a conversation in real size but you can also scale it up and down as much as you want.
12:46
Like in the lower right image you can see that the girl is standing on the pedestrian in front of the guy. Very small in size. You can actually have virtual people running around your table while you do other stuff while talking to them.
13:05
So another very classic example I would say is AR navigation. This is a system by Google which is a direct advancement over their smartphone navigation. And I think despite being a very obvious example of when you think what AR can do for you it's also a very powerful one.
13:25
So basically it means like when you want to go somewhere you project like for example a line in front of the people that you have to follow to reach the destination. It's very easy and very understandable. Currently there are mostly specialized systems that focus on something like airport navigation or car navigation for example.
13:48
Very few for general pedestrian use but I think that's likely going to change in the future. So the next one also maybe a little obvious.
14:01
Finding places of interest based on selected preferences. Example would be like where's the next post box, where's a good restaurant or bar. So this is of course possible today with smartphones as well. But the big difference is that you don't have any abstraction. So you don't have to match the map of what you're seeing. So where is this building that it says on my smartphone.
14:24
You get the information placed exactly at the point of interest itself. So that makes it a lot easier and useful for example. Of course you can also show something like user reviews or for example the menu of the restaurant so that you can find a suitable match.
14:44
What's also very very useful is something like AR translation where you provide an instant translation of a foreign language in place. So for example you look at a sign in Russian and you get the content in English.
15:01
This is actually Google translate which maybe some people already have used. When you have used it you would say oh well that's really practical for example. And when you use it with a head mounted device that's even more practical because you don't actually have to point a device somewhere.
15:21
You just see the text and the language you prefer. But besides translation you can also offer context information as well of course. Because you can not only show in this example the direction but you can also show the distance to the object. And it's also suitable for places like museums where space for text is usually very limited but space in AR generally is not.
15:51
Which brings us to the next application case which is cultural heritage and tourism. And this is a nice paper that gives a good example of that because it can make history somewhat more experienceable.
16:07
So when you have ruins for example of a Greek temple like here you can get an impression of what the temple might have looked like directly situated on the actual ruins. Or when you have the fields of antique running tracks you may have virtual
16:23
athletes in augmented reality that give you an impression how these facilities were actually used. Like the people running around doing sports and stuff like that. And I think this is really powerful I think you just get to see what it looked like in the past.
16:41
But another important aspect and I think a very powerful one is how AR may have the power to make technology more accessible again. So today's technology grows more complex and it's more and more connected so that even
17:01
for experts it's sometimes hard to actually say okay what state is the system currently in. For a layman person it's even harder. So AR can help make this hidden information visible again in a way that's easy to understand. So just to give you a little example which is not perfect but hopefully sufficient to transport the idea.
17:22
Imagine you have a simple router at home for example. You have some status LEDs but when for whatever reason your internet breaks down or stops working they don't have you that much for debugging the actual problem. So to get more information you have to most likely access a web interface if you still can remember the IP of your device.
17:42
Of course you can do what we have done here and put a little piece of paper with the IP on the device itself but that's hardly a practical solution. So imagine with augmented reality you can just look at the router, do a simple gesture and then get the information you were interested in directly superimposed on the physical device itself.
18:00
So which can help you immediately recognize what the problem might be. So maybe you just forgot to turn on DHCP or something like that. But we can expand that concept to not only include a router but also the wireless devices connected to it. So one of the predominant features of wireless LAN is that there is no wire so it's hard
18:22
to see which devices are connected, where they are located and what the IP is and so on. So where they are we can simply draw virtual connections to make that information easily available to the user. So I do something like this. But we can also use it to make errors more recognizable.
18:42
For example one of the devices maybe not be able to resolve websites because there is no DNS server registered. And then we can of course also display additional information like the bandwidth a device uses. And when we have an actual physical medium we could actually use that for displaying information like a graph of the bandwidth over time.
19:04
And it's very easy to understand what is displayed because it's directly located at the medium itself. So I hope this example gives you an impression of how AR can really help with understanding technology again. We might talk a little bit about that later on.
19:22
But first let's come to some technical challenges. So how does that actually work and what has to change so that we can really use it every day. The first one is tracking which is one of the main challenges because I have to locate the user somehow. I have to know where he or she is and what she's looking at for example.
19:45
So for general positioning I could use something like GPS. But precision is of course a problem and I can't really use it indoors. So we need additional tracking technology. And there are many different approaches but the current state of the art is more or less infrared tracking with time of flight.
20:05
So that means infrared light is emitted which is reflected by the environment. And the time between sending and receiving the light impulse determines the distance to an object. And with that I can create a depth model of a room which more or less looks like this.
20:23
And then I can not only tell where the user is right now but also when he or she is moving around. And while this technology is far from perfect I think the current tracking technology is quite robust and useful in that regard.
20:41
What's also important is display and rendering. So there are many different technologies for head-mounted devices. One of them is video see-through where a camera films the environment which is then together with the virtual content rendered onto a display in front of the user's eye. But latency is a very big problem in that regard because you will very easily get sick.
21:05
Another is see-through objects using polarized glass where the air content is projected onto. And there are even small laser projectors which project directly into the user's eyes. But for some reason user acceptance is not very high.
21:24
But the ideal display would not require any external display at all. But you would directly manipulate the optic nerve which sends the messages to the brain. Well we don't have that right now so we have to make through what we have.
21:41
And this is basically see-through optics which is right now like the current state of the art. So let's look at what the problems are in that regard. One of the problems is opacity because when I have a solid object like a human for example, no background should bleed through them.
22:03
So currently for the commercial headsets this is not so bad. But really depending on the lighting conditions when you have a lot of sunlight you can't really see anything anymore. But there's still much room for improvement in that regard. What might be even more important is the field of view of the AR headset.
22:24
Because like a human has roughly 210 degrees field of view. And as you can see here like the Microsoft HoloLens has like 30 degrees and the Meta 2 has like 90 degrees. Most of which from my experience at least are pretty blurry.
22:41
So yeah that's not, there still has to get a lot better until we can really use it. Also very important is when a person wearing an HMD, what does she see?
23:01
Like how can I recognize what a person is looking at and how can I get data on the recognized objects? So due to performance requirements most of the processing will most likely not happen on the head mounted device. But it will very likely take the form of some web service or similar services.
23:25
Which not only process a constant visual input stream of the cameras of the HMD but also have to provide data sources for the various objects. For example when I have touristic points of interest I need to get some information on them.
23:41
So this requires a constant data stream and a very good connectivity to make it really useful. And it also requires a lot of services readily available to use by the AR device. So this is quite a big challenge, maybe the biggest of them all. So to give you a little impression how that all could work based on existing research which is called a reality based information retrieval.
24:07
So imagine you have an AR HMD that feeds a constant live image to a server to recognize objects the user is currently looking at. For example vegetables at a market. So the server is able to classify those images by comparing them with a couple of saved reference images.
24:24
And gives a list of recognized objects and the confidence value back to the HMD. And then the HMD can query a different service to get some recommendations for meals that I can cook with that vegetable.
24:41
So when I explore a market and you have your AR display with you, it could not only tell you what a particular vegetable is, in case you might not know. But it could also tell you what to do with it, which more or less could be quite useful. What's also important, if AR glasses are to become everyday objects, economics and aesthetics are going to be important.
25:11
So you probably don't want to wear something like this right now. So for the most part that means shrinking the glasses down to the size of current normal glasses.
25:23
Which is a form factor that's more or less already accepted within our society. And as you can see in the comparison we are not really close to being at that point. And one of the most important questions is not only how to display the AR content, but also how to interact with the content.
25:47
Because otherwise it's a very limited experience. So while AR objects might appear to be part of our environment, we can't really touch them. So I can't actually press a virtual button.
26:02
And there is research concerned with providing haptics as well. For example there is ultra haptics, which uses ultrasound to create resistance in mid-air. So that I have an actual resistance when I touch somewhere, but it's still in a very early state. So currently mostly gesture interfaces are used, where my fingers are tracked.
26:25
And the line of sight together with the hand gestures are used to trigger interaction. Speech input is also a common solution, but use in public environments is limited. And another possibility is also to use additional devices like specialized controllers.
26:45
Or for example the touch screen of a smartphone for interacting with the virtual content. Gaze tracking, that means tracking the user's eyes, will most likely also play an important role. But right now there is pretty much no perfect solution.
27:01
And also still a lot of research happens in this regard. So this is a very hot research topic right now. So let's come to possible dangers and other issues. So all the things that might not be so great about ubiquitous augmented reality. The first one, pretty obvious problem is information overload.
27:22
Where too much information leads to excessive cognitive load by users. So virtual content has to be used sparsely to best support the users. And the only way to make sure users are not over-strained is to leave them in control of what and how much is displayed. But users might not always have the means to control it.
27:45
So especially when third-party services are involved, we may see something like invasive advertisement and other unwanted content. And since head-mounted AR is already a personal experience, these ads are most likely to be personalized as well.
28:06
So this directly leads to the question of privacy and data security. There will most likely be a constant stream of the user's current position and a live video feed to various third-party services.
28:21
So this raises the old question, how can I control what happens with my data and who gets access to it? And privacy of course is also a big issue because there's hardly anything more personal than a constant live feed of what I'm seeing. So even if I myself decide to don't use an AR device, as long as everyone else does, it hardly makes a difference.
28:46
So at the center maybe is really the question of who controls the actual hardware and software platforms. So similar to smartphones, it's likely for a few global operating corporations to not only offer the AR devices but also provide the corresponding software ecosystems as well.
29:03
So we see that already with the direction the HoloLens by Microsoft is taking. AR consists of even more sensitive data than current smartphones. So will we be in control of our own devices? Do the companies producing them have any interest to keep us in control?
29:23
Because most likely their primary motivation is to make money. And will people even care? Because they hardly seem to care with smartphones right now. So I think this is an important question. And another possible application area for augmented reality which I haven't talked about is military use.
29:45
So heads-up displays have been used in the military and airplanes, for example, for over a decade. And there also have been several scientific papers evaluating the use of augmented reality in military use.
30:00
So it's not difficult to imagine AR systems like telling soldiers where to go and maybe even who's friend or foe. Microsoft has recently made a contract with the US military for over 480 million US dollars to evaluate the use of the HoloLens on the battlefield. And the goal is to increase lethality, mobility and situational awareness of soldiers on the battlefield.
30:29
So I don't really want to provide any judgment on that. But it's best to not have any illusions what this technology can and eventually will be used for.
30:41
And it's probably not all going to be unicorns and rainbows. So let's keep that in mind. Another interesting aspect is how does massive use of augmented reality actually change how humans interact with each other. So for example, when we can project ourselves everywhere at any time, how important will it be to meet in person?
31:05
So to give an impression, there's the novel Naked Son, written by Isaac Asimov in 1957, where it's common for people to communicate with photorealistic holograms and even being naked in front of each other. There's no problem, but it's extremely unpleasant for people to be in the same room with a real person.
31:25
So I'm not going to say this is what is likely to happen, but I think it's an interesting aspect to think about, as there will be some social changes involved, which we don't know yet. This also leads to the question how our own way of thinking is influenced by the constant use of augmented reality.
31:46
So for example, how does our perception of locations and how locations are connected change when we are guided by AR navigation, which in the worst case means following a line until we reach our destination.
32:00
There has been research showing that extensive usage of GPS might lead to an atrophy of the hippocampus, increasing the risks of cognitive diseases like Alzheimer's later on in life. So it's important to be aware that there might be issues and it's important to do the actual research to figure out what these issues might be.
32:24
And of course, maybe even the ultimate question, what is reality? How does our understanding and perception of reality itself change when we use AR? This is more a metaphysical question, but I think it's an interesting one nonetheless.
32:43
It's not so much a problem for moderate use of augmented reality, for example, to annotate objects, but more a question for photorealistic augmented reality, which can alter the way the world looks to each person individually. So if my world looks vastly different from your world and when AR is always active,
33:05
what does reality really mean in this regard and what role, if any, does the unaugmented world play in the lives of people anymore? So this might be something to think about. And with that, I conclude my talk.
33:20
So hopefully I could show you that augmented reality could have a benefit in our life, but that there are also some risks involved and I hope you found it somewhat interesting. And now we can have some questions. Thank you.
33:43
Thank you for your fantastic talk. Good overview of whatever is possible in here. I see some people here queuing to ask you some questions. Let's go to number two here, please, sir. Thanks for the talk. The previous slide on the red and blue pill, can you elaborate a little bit on how you as a researcher and user of augmented reality eventually better understand reality?
34:11
So you mean when we use augmented reality that might lead to a little better understanding of what reality is? Or what did you mean? Yes, that's the question.
34:21
Do you feel like you're not just using but also researching? Because in order to, for example, when you have this slide on how the 3D scan works, you use it as, let's say, a metaphor or a shortcut on how our own brain and visual system works? Yes. Well, like I said, it's a rather metaphysical question.
34:41
So it's hard to really answer the question right now. It's more like you could guess how it might change for people, what their perception of reality, how does that change? But I can't really answer that, to be honest. It's more of an open question for myself as well,
35:01
because when I think about augmented reality and if it eventually will be used by everybody every day, I can think of some dystopian sci-fi scenarios where everyone lives in this happy world which is totally fake, and the real world basically looks awful but nobody actually notices that
35:24
because everyone is caged in his own happy place, for example. But I don't know if that's going to happen. It's just a possible danger that you could think of. But this is probably something that will not happen in the next 20 to 30 years because we simply don't have the technology to do that right now.
35:43
So maybe that more or less answers your question, I hope. Next one. So to get widespread consumer adoption, we really need the miniaturization of the hardware.
36:02
Google Glass got kind of laughed out of the room a few years ago, so we need to be better than that. Yeah, but Google Glass wasn't really augmented reality. It was more like a heads-up display, because you had this tiny screen in the upper part of your field of vision, and it was like you could just display a limited amount of information there.
36:24
When you have real augmented reality, it's basically having the whole space to put information. Sure, I mean more like it's a stupid-looking thing to walk around on your head. So we've got to look better than that for the mass consumer adoption. Yeah, but that's why I think that we have to go to where medical glasses are right now,
36:44
because they are already accepted in society. So someone wearing glasses doesn't get looked at in a funny way, because it's pretty normal. So if augmented reality headsets look more or less the same, I think we're fine. So the question is what are the technological problems that we have to solve to get there in a consumer device?
37:02
Yeah, we have to actually miniaturize the stuff. So I think the laser projectors actually are an interesting solution, because they are rather small, and you can project directly into the eyes of the users. So if we get this user bias figured out, how we can actually convince people that this is a good idea, not dangerous at all,
37:20
we can shrink it down even further. And if we have the whole processing, not on the device itself, but on third-party data centers or something like that, we can also shrink that down, so we have to actually only stream the input to the server, and we have to stream the image back to the device. Then we can build pretty small devices, I think.
37:43
So that's what we have to figure out. But yeah, I think this is something that's actually going to happen, I don't know, in the next five years or something like that, because this is actually something that's worked on right now. Microsoft actually is building the next version of HoloLens, which should be smaller, which should have more features like eye tracking and stuff like that.
38:03
But we have to wait and see, basically. This is also, I think, an important thing. The hardware involved is very complex. So it's not something that one person can do in his garage or something like that. So you pretty much only have the chance to have those big corporations do the stuff and do the research,
38:23
because it's so complex. With a little bit of imagination, we see someone stepping in there from the Internet who has a question. Holographically projected here next to me on stage. The Internet has a related question. How long do you think will it take until everyone uses augmented reality like mobile phones today?
38:48
I think that really depends on the use cases. I think this is the most important thing for acceptance. You have to provide functionality that actually benefits the way people live their lives.
39:01
For example, the AR translation, I think, is a good example, because it's common for people when they are traveling, for example, to run into signs or something like that, which they can't understand, because it's a foreign language. And if you provide an augmented reality solution for this particular problem, people are going to use it, because it's practical, because it's useful.
39:21
And the more functionality, it's the same with smartphones, really. The more functionality you provide that goes in this direction, the easier the adoption will be. So, yeah, that's basically, we have to find use cases which are important to people, and then the adoption will basically happen itself.
39:41
Critical modes, probably. Yeah, critical modes. Number three, please, sir. Thank you for your talk. When you introduced the platforms, what is with Facebook with the Oculus Rift? Is it not virtual reality? Yeah, it's virtual reality. It's not really augmented reality.
40:00
I was thinking about including it, but I still don't have any idea where they are going with this. So, they have bought this company, they have invested in this technology, but I don't see where they are going with this right now, what their goal is, actually. So, this is why I didn't include it. Thank you. We will come in your direction anyhow.
40:23
Number two, please. First of all, thank you for the informative talk. And what do you think will change from privacy policy when you have an active live feed of what you're seeing with augmented reality? Yeah, I think this may be the cynical way of looking at it,
40:43
but I think people will stop caring about that. That is what simply is going to happen. The current generation is concerned with privacy, and we feel like, for us, it's a very bad idea to actually have a constant live feed of what we are seeing transmitted to somewhere on the internet.
41:02
But I see that with younger people who use smartphones. They don't think about privacy pretty much. They think about what this device can do for them. And I think this is more or less the social change. Maybe that will happen. People will stop caring about it. This will just be a normal thing that your field of vision
41:20
gets transmitted to somewhere on the internet, and people will simply stop caring. I think this is what's going to happen, yeah. Thank you. Okay, next, number two. Thanks. I saw recently there was a company that announced some shades,
41:41
and they had a bone-conducting sound transmission. So they didn't really project something into the retina, so they went away around all the projection and all the complexity of making this huge, bulky headset. Do you think this is a viable way to go forward? You've focused mostly on visual inputs,
42:02
but maybe other sensors are better suited to making an entry into augmented reality. What do you think about that? An interesting point. I focus a lot on the visual side of things, but the important thing, I guess, with all this mixed reality and virtual reality stuff
42:21
is that it's not limited to the visual. So basically it should be a whole sensory experience, so all your senses are involved. So yeah, maybe that's a good idea to not only focus on the visuals so much, but also maybe on things like audio, like you said.
42:40
It's really the same question with the use cases. When you figure out a very good use case where an audio augmentation helps people in their everyday life, I think people will start to use it. The reason why I focus mostly on the visual part is because I think it's the most strongest sensory experience
43:02
people usually have, when they have eyesight at least. And I think this is most likely also to be like the selling point of augmented reality, will most likely be the visual part. OK, another one. Thanks for the talk.
43:21
You mentioned that one of the ways you see the miniaturization of these devices evolving is by streaming data into the cloud. But we've seen, especially with smartphones these days having the equivalent computational power of computers from a decade ago that it is actually possible to do a lot of these
43:43
machine vision tasks on a smartphone. I think some manufacturers are basically trying to get your smartphone to be your primary computer. So do you see that as a likely alternative? Obviously this comes back to the privacy question because if you can keep things local then...
44:02
It also comes back to what the producers of the devices want you to actually do with the devices. Of course when you have miniaturization that is powerful enough to put a whole computer into something like normal glasses, you could do that and you don't have to stream that much information.
44:23
But I guess you still have to actually do all this stuff because you are not alone in the world basically with augmented reality. So if everyone else is using augmented reality you need to have that actual data from what the others are doing. For example, in this building
44:40
I have to build a constant room model of how this building looks like. So everyone who is moving through this building will constantly scan the environment to build the 3D model of the environment. But you have to share that information so that when I move into this building for the first time I still have the whole environment mapped out
45:00
because someone else already walked through it. So you still have to stream all that and you still have to share all those data. So I think even if you increase the processing power of the devices itself this is not going to change much. My question is very much related.
45:21
With all the big players designing the devices it's still a closed world and I would have to imagine I would have to stream my live feed to some Microsoft server or whatever. Are there any efforts to keep at least interfaces open if not the whole software open source?
45:40
So for example the CCC could put up a server I trust and send my live feed to. The system is open or not you can actually change the location while your information is transmitted to. At least I don't know of any open hardware open software projects
46:00
who try to do Augmented Reality right now. Of course there's research. Research is more or less open most of the time but research prototypes right now while they are good at what they do they are hardly like consumer products. So you can't really mass produce them and distribute them to the people. So yeah, you probably could do it
46:21
but I don't see currently I don't see any movement in this regard. I shouldn't do that but just for the previous question I would suggest Speak up please. I shouldn't do that but to answer briefly the previous question WebAR and some efforts from Mozilla are trying to bring at least
46:41
the highest level of the stack in an open source fashion for Augmented Reality. So something to check. Yeah, I was wondering for the example you showed on your research in your own lab about information retrieval those examples that are able to distinguish between one object another, let's say an apple and a pear or a tomato works well
47:02
but I'm wondering how scalable it is because when you get to a more precise example Yeah, that's a good point because what we were using was an actual food database for that so it was very easy to classify the vegetables because they are food but if you have a database with everything in it of course it's a problem
47:21
because classification will get you some weird results so maybe the vegetable would be recognized I don't know as a lamp or something like that. Yeah, that's a problem. But this is only a problem like in this particular research case it was more like a proof of concept to actually show how can we use Augmented Reality
47:42
in some more practical everyday scenarios it wasn't really much focused on to find a perfect technical solution providing the classification of the images this is also why we use an existing service for that and we didn't build our own so we just used what was available.
48:01
Okay, unless someone is beaming themselves up from the internet then we have all the questions here answered thank you very, very much I hope to see you soon
Empfehlungen
Serie mit 18 Medien