PlotVR - walk through your data
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 118 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44808 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
EuroPython 201963 / 118
2
7
8
15
20
30
33
36
39
40
45
49
51
55
57
58
63
64
66
68
69
71
74
77
78
80
82
96
98
105
107
108
110
113
115
00:00
GoogolPoint cloudRevision controlComputer programmingBitScheduling (computing)Computer animationLecture/ConferenceMeeting/Interview
00:22
Client (computing)Multiplication signPlotterComputer animation
00:58
MathematicianPlotterFile formatSlide ruleProjective planeInformation technology consultingMultiplication signOpen sourceLink (knot theory)
01:34
Information technology consultingGroup actionVisualization (computer graphics)Power (physics)Service (economics)CognitionServer (computing)Information technology consultingPlastikkartePlotterHome pageClient (computing)XML
02:20
Mobile WebMatrix (mathematics)Android (robot)outputDimensional analysisMobile appClient (computing)Workstation <Musikinstrument>Three-dimensional spacePlotterBasis <Mathematik>Point (geometry)Library (computing)BitMobile WebProjective planeSound effectGoogolComputer programmingRevision controlOpen sourceGraph (mathematics)LaptopComputer hardwareMultiplication signField (computer science)Real numberComputer animation
06:35
Matrix (mathematics)Android (robot)outputRevision controlRoundness (object)Computer animation
07:02
Demo (music)Addressing modeIRIS-TView (database)Kernel (computing)Computer filePrice indexProxy serveroutputTouchscreenCodeMobile appServer (computing)Process (computing)PlotterIRIS-TSet (mathematics)Computer animation
07:50
CodeView (database)Computer fileKernel (computing)Price indexWindowStatisticsoutputClassical physicsRevision controlServer (computing)PlotterFunctional (mathematics)Wireless LANComputer animation
08:25
GodRevision controlServer (computing)Demo (music)QR code
08:52
Computer fileKernel (computing)View (database)CodePrice indexFile viewerPlanningComputer clusterDifferent (Kate Ryan album)Order (biology)PlotterKeyboard shortcutMultiplication signRevision controlProjective planeTwitterIntegrated development environmentWorkstation <Musikinstrument>BitComputer animation
11:32
CodeView (database)Kernel (computing)Computer fileKeyboard shortcutFocus (optics)Demo (music)Plot (narrative)Multiplication signMereologyKeyboard shortcutFirst-person shooterDemo (music)Focus (optics)Software testingType theoryBitComputer simulationComputer animation
13:38
Computer simulationMereologyComputer animation
14:07
Computer animation
14:53
Set (mathematics)Computer animation
15:14
Direction (geometry)Computer animation
15:43
BitDemo (music)Slide ruleGodComputer animation
16:21
Demo (music)Web browserGame controllerServer (computing)Java appletPlot (narrative)Software testingLink (knot theory)Execution unit3 (number)Slide ruleGame controllerNumberIP addressMobile appServer (computing)Connected spaceBitRevision controlWeb 2.0Broadcasting (networking)Workstation <Musikinstrument>Socket-SchnittstelleFocus (optics)Network socketComputer architectureMechanism designoutputLaptopComputer animation
18:17
Software testingLink (knot theory)Android (robot)Installation artoutputMobile appRepository (publishing)Data storage deviceNumberXML
18:45
Plot (narrative)Software testingProxy serverLink (knot theory)Machine visionLine (geometry)PlotterCommunications protocolAuthenticationAuthorizationServer (computing)Interface (computing)Mobile appCommunications protocolWeb browserAndroid (robot)AuthenticationVector spaceRepository (publishing)AbstractionSet (mathematics)Link (knot theory)Line (geometry)TrajectoryWordAuthorizationServer (computing)Interface (computing)Demo (music)SurfaceData storage deviceSynchronizationProcess (computing)Computer animation
21:04
Projective planeOpen sourceElectronic visual displaySoftware development kitProteinDirection (geometry)Revision controlMobile appIntegrated development environmentQR codeData conversionPoint (geometry)Firewall (computing)Software developerArithmetic meanInterface (computing)Augmented realityData structureView (database)VolumenvisualisierungRoundness (object)PlotterWeb 2.0Medical imagingRight angleBeta functionMultiplication signWeb browserDatabaseSoftware testingFirst-person shooterWebsiteComputer animationLecture/Conference
Transcript: English(auto-generated)
00:02
Yeah, so thanks guys, how are you? Like it. So it's my first Europe Python, but I like it a lot. Let's see where it's next year For those that so in the printed version of the the program now was something about cities That's not it. So they changed the schedule a little bit. So yeah
00:25
Yeah So I'm gonna talk about plot. We are a client kind of a hobby I had for the last couple of years Now then doing something for it And now that I was accepted here for Europe eyes and I finally could put some effort into it to have it like
00:42
Usable and since yesterday evening it for the first time it did go through so let's hope so in the morning today It wasn't maybe In the meantime it was so, let's see It's not as polished as it could be So maybe about me first
01:02
Yeah I'm a mathematician by formation spent a couple of years in academia Now since two years, I'm senior consultant with the one-shot solutions where I'm doing projects in data science AI coaching people setting up infrastructure for data science stuff like that and also
01:22
Having I'm having some open source projects. So you will get the slides there. You will find the links and Introducing today for the first time to a public audience plot VR is there as well? One or two sentences about the one solutions
01:40
so yeah we are about 40 consultants and I think it's I It's really correct to call us a talent heaven. So if you think you are a talent and You're working or want to work in Switzerland, then maybe contact us we are hiring
02:01
Come to me or go to our home page. We are doing all the things for clients in the data pipeline Yeah, and Yeah, it's really a Smart team that we have there Okay, what is plot we are about so
02:23
Show of hands who of you is doing like graphs like this or plots scatter plots stuff like that on a weekly basis Or monthly basis something like that dashboards, okay Lots of people data scientists do that a lot and you you really get skillful and producing the things that you need
02:43
but now and then You are missing this one dimension that actually would be there the third dimension Yeah, you know and then okay you go you look up what is what is there in the in the field? What are they have libraries and you probably have used them? so Who has used such like plotly or or scatter 3d or stuff like that?
03:04
most of you okay who is using them on a as Well weekly or monthly basis Okay one person there Cool I Usually don't stick with them. I
03:22
Think it is because they are still they are not the real thing They are still a 2d projection of the data. You don't really can feel the data the third dimension So there are they used to be expensive possibilities like 3d Monitors or maybe domes stuff like that
03:42
But as data scientists you usually it's not so important for you to use these tools So probably you won't have standing something like that around So three years three or four years ago there was this announcement At Google IO about the cardboard and when that came out, I thought hey
04:03
If that's inexpensive so because you basically you need to buy the cardboard you don't even need to do that and then you can Visualize really in 3d and you get a three-dimensional impression because you really have to stereo effect. That could be it
04:20
so Yeah, I thought hey, let's do that. And that's what this talk is about So, how did I progress with that or what are the issues you you need to think about so the first issue is okay, you are on your desktop or notebook having your data doing your ggplots or your
04:43
Seaborn or whatsoever and now the idea would be to get it onto a device So you need to somehow transfer the data from from your from your workstation to your device Okay
05:00
Okay, that's one point you need to think about probably Bluetooth could be an idea but that's a little bit weird for me to program I have Tried it maybe once but I didn't achieve it. So I thought okay, let's do it via HTTP stuff Then maybe we can do so two years ago. The AR hype came so maybe you can do both of those together
05:22
That would be an idea then the next question Can we just put like a mobile web app where we look into these things or do we need a real? iOS or Android app programmed Also, how do we navigate in this VR That's always a problem
05:40
And then will it cost to use or something? So will we need some some expensive hardware or not? So the idea was no, let's use open source for that And then also can I create all of that in my free time? So the last point was maybe the the biggest issue. So that's why it's now three years later
06:01
So a little bit an overview what what is there now So basically we have three Three devices where we cut our three clients where we can look into the HTML one the iOS one and the Android one The Android one I Started back then when I did those things and then I switched to an iPhone and now I'm I did it mostly for iOS
06:26
So the Android is a little bit lagging behind Now VR works in all of them the HTML version of the VR is kind of not nice. I don't know why The AR version of HTML seems to not be as nice today. So it needs to have like some some anchor
06:44
in your In your vicinity But the others yeah, you see what it is there and actually Since today also the iOS version is on github So if you want to double around with it Yeah, let's show you how it works
07:03
so I'm going now for the For the iOS up, so, let's see Okay, so here you see my screen share and here see say the Jupiter lab so the code basically It's really simple You import
07:21
This package after you have installed it This will start in a different process so a tornado server on this port 2908 and Yeah That's it now you need some data to plot let's plot iris who of you doesn't know what the iris data set
07:45
No, so these are the not dying data scientists probably I don't know or not the classical statistical data scientists Okay, and then you play basically send it to that server. So now okay, that could be just empty functions, maybe
08:03
Let's see what happens now in the iOS version So we open this plot. They are now you see they need to communicate via IP So they are actually I'm using here my phone's wireless also here for for my
08:22
MacBook and It actually shows me up that here. I need to connect to that. So first of all, I need to find the server so I'm Going for the QR QR code tech. There it is So now it's there. So I now want first to look at the AR version
08:43
So, let's see what the demo gods think Okay, so first of all, yeah microphone is maybe not there so you see here I put like the origin it's always where the phone is when it started and now you can really
09:01
Go and look into your data Between having like a 3d 2d plot or 2d projection of your 3d things So below you see the where it found like the the plane the floor
09:22
So in an advanced claim it would like calculate is it a floor or a desk and put it in different heights Stuff like that. But yeah, so there are some issues if you go like this you see from from behind It's dark because I don't Yeah, but I think this is nice Is it okay for you if I take a picture of that?
09:44
With you so and can is it for who is it? Not? Okay, if I put that say on Twitter or stuff like that Okay, then cheese Good, so that's the AR version
10:01
Now for the VR version Let's see now I will need to Do like this and this and this So here the problem is I don't know whether we can maybe we can kill the lights or at all
10:21
I don't know whether that's possible So you will see it only just very faintly now In the in the middle here, I don't know whether you can see it. So yeah, that's the first thing a
10:41
Little bit, huh So yeah, that's the problem. Now. We would need to get it closer to us No the data in order to be able to really see it So, how do you navigate and now it usually this cardboard thing? You would have it like really here and you wouldn't see anything else Yeah, so you need to have something in your hands or in one of your hands and that you kind of know
11:06
So something like a gamepad so you could connect a gamepad to that thing, but probably most of you I don't know Maybe I don't euro Python more than in other Circumstances don't have a gamepad in the in their working environment
11:22
So well, let's think about what do we have ah Yes, we have the workstation oh there there is a keyboard so Could we use that? I mean here we are typing all the time. Yeah Okay. Yeah, so we just need to get those keystrokes and push them to the phone. So let's do that
11:49
Maybe do it a little bit bigger This and now it needs to have to focus of the of the keyboard. So we click on it
12:00
Tada and now you see you are like in who of you has ever played like a first-person shooter game So, you know a SDW probably so now if I I Never know which way around I did it because I couldn't test it that much So I'm pushing and now
12:23
Getting the data back to me. So I'm clicking on s so now I need to get it up Yes Now you should see it much better. No, you don't see it at all need to
12:48
like that so again, I Can get it up I can push it back Down come back to me and now I'm through so I need to get back
13:02
So basically if you know your your first-person shooter, and I guess you you do blind typing in first-person shooters You don't w s so, you know that by hand So if you are in the immersed situation You really can go through it and and really start examining all the things in 3d in VR
13:23
Okay, so that's basically the demo part so just one For debugging purposes because that's the only thing that also works in the simulator. I have also the 3d part here
13:45
And you dare really see okay, you can get to the data and you can explore it But it's for sure not as nice as if you are doing it maybe
14:13
Okay, maybe go for you some of you might have come for the city's talk so let's do the following
15:07
Maybe even do it like this So you I guess most even if it's a international conference most of you might recognize this data set
15:20
It's Switzerland But if you now go close to it So you really can start But use that on the using that on the iPad is really amazing. So I like it a lot so
15:47
Just to finish a little bit
16:06
So, yeah the demo gets God's were willing so I don't need to show that slide oh It's not connecting them. Oh, so this slide is not needed, but I I hope you agree. It works kind of
16:33
So a little bit the architecture now is is really simple we have a Server in tornado that can run anywhere basically usually it runs on on your workstation where also your Jupiter runs
16:46
In the Jupiter notebook, you have an iframe that embeds this HTML controller you can so from the Jupiter the data gets posted to the To the server the iframe here the controller talks via web socket to to the server
17:04
To exchange to send the keystrokes and to exchange status So, you know whether how many controllers are active and so on. So basically here you see There's one device connected. There's one controller and it has focus. So if you go If it loses the focus
17:22
Yeah, then the focus number goes down something like that it would be nice also to show it then in in in the app And basically the server then is a broadcast mechanism so all the the devices connect get the data right now we are HTTP get and get the keystrokes and the status via web sockets and
17:44
Right now be careful. Everything is unencrypted and it's open to everybody who knows the IP address and the port so That's for sure something that if it gets bigger one needs to address Okay
18:00
Do you want to have it? Good, so you can pip install it right now from from github Currently, I hope that I push it to pack a soon. You can get it for our it's a slightly outdated version right now I will have that up in in couple of weeks the iOS app since I think two hours is on github
18:25
Either you download that you clone the repository and install Xcode and you Then you need to have a free personal team setting there and then you can deploy it to your devices I think I will have soon something where I can push it like to a hundred devices worldwide if they gave me give me
18:42
The device number and then maybe via the app stores test flight I can even have like a shared public link eventually, but there it needs to go to the browser So I think the app is not ready yet And the Android app is there as well. So there are apks on the releases Here as well slightly outdated. Yeah. Okay, so
19:05
What do I think? Will do Will be the next step. So first of line first of all, I would really like to have texts there So my my dream demo for today would have been
19:20
If it would have been running already say day before yesterday Say to get all the abstracts of the talks here Puts them through word to back so that every every talk gets like a vector And then you can walk through the talks. So that would have been a really nice demo
19:42
It's not that hard to do now Yeah, I hope that so stay tuned for that. I'm also having lines or surfaces I don't know the ISS trajectory to see it really in 3d would be something I'd like to do or a flight Then make the scenery nicer maybe don't have a black background or I don't know
20:04
Put it onto all the Repositories and stores and then solidify the protocol and make everything like really working together. So that's Maybe I hope that by end of summer. I'm somewhere there Yeah, and then there are some other ideas HTTPS having that having something like an
20:26
Authentication authorization Actually Jupiter is a tornado server. So could we just plug into it? But I think it's it's not that easy You need to be careful with the async things. Maybe then it works
20:41
Interface to plotly ggplot then and maybe eventual have like some some new Data Hub where you really can walk where you can publish your data sets and people can walk through them stuff like that That would be really nice. So Yeah, thanks to all the Giants shoulders. I'm standing on
21:03
Also my employer To send me here Also to Europe isin to accepting this talk so that I finally could bring it forward and to all the future contributors of these open source projects Thanks
21:24
Okay, I think we have a couple of minutes for questions. So there's a microphone over there. I've got one here Anybody has any questions? Yeah Do you have an idea how to get users for that? Like, you know, like you're talking about
21:44
Technical stuff right now, but it would be like really nice that like lots of people are using it. So there is I Once worked like with proteins and stuff. So we're just like online database with 150,000
22:00
Proteins like it's like 3d structures and we have like a lot of tools so to plug such finger on this is like In my eyes, it should be like pretty nice because people are using it not here. That's probably not in your workplace. But yeah Biologists all over the place. So yeah, that's I guess an idea and probably a question, but I don't know
22:24
Yeah, thanks. Yeah, I think actually this this isn't like constrained to to data science plots I mean having something where people can by some means produce some 3d things and
22:40
Just being able to look at it in their devices and I mean Apple really is announcing now in a beta version Yeah, what is it called reality composer or something like that wherever you can start building things to your AR kit things I guess that's the same direction. I'm more now on where I'm coming from. So
23:09
I Can use it for for many things so Yeah, so maybe if you if you're willing you can write a converter that just gets out the the data points and
23:23
Then we we can just grab all of those proteins and Publish them in a in an AR view. Maybe they haven't done that yet and for us now It would be just the conversion then having it Yeah, thank you for your talk
23:44
So my question is have you actually tried it in-house for practical things and what were the results because With these first person 3d shooter, we know that this Experiences can be pretty immersive and well, it's kind of we navigate it from the technical side Of course is bleeding edge of interfaces. It's like very interesting technology to play with but I'm interested more on like perception side or
24:07
Neuropsychology side of these things are they really make people work and analyze this data more efficient Yeah, we will see so that's the test of time. No, I think actually the AR thing is
24:22
nicer for that because Really the VR thing it's too Too occlusive so you are really in in the data and it might be something cool to do but maybe not that practical I think the AR thing is much more if you really can just push it onto your phone and then look at it
24:43
Like if it was standing there on your desk and you can look at it and if it's no hassle So that's that's the key point. It's no hassle if it's no hassle I think then it it could be really beneficial. Yes. I mean in in corporate environments. There is the problem of
25:01
Do you get the data from your? Through the firewall to your device, I guess there will be a problem So I'm not sure how to address that yet. Maybe one can push it through through the QR code actually now that I'm thinking of so there's some
25:24
Developments that allow browsers to render augmented reality images there So you are talking about web XR for instance, yeah, yeah, so there the AR needs really like a barcode Anchor, so I mean we could just have that on the display and then it's like anchored to the display
25:44
Could work I Kind of don't like that too much personally I don't know why so I thought I really want to go for the app there Because there it's really you don't need that anchor so you can also show it then at lunch to somebody or
26:04
show your newest plot to your significant other in the in the evening, so, you know But yeah, I mean that that certainly would be to have it In an HTML version. I think that would be the way to go right now. Yes
26:24
Okay, any other questions? No, then I would like a big round of applause