pyRT - Computer Graphics in Jupyter Notebooks for Fun and Teaching
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 130 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/49989 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 202014 / 130
2
4
7
8
13
16
21
23
25
26
27
30
33
36
39
46
50
53
54
56
60
61
62
65
68
73
82
85
86
95
100
101
102
106
108
109
113
118
119
120
125
00:00
MathematicsTouchscreenElectric generatorPerfect groupLaptopComputer graphics (computer science)MereologyShared memoryCopyright infringementQuantum stateMultiplication signMedical imagingUniverse (mathematics)GeomaticsMeeting/Interview
00:55
Computer-generated imageryGame theoryContent (media)Complex (psychology)Real numberDemo (music)ProgrammschleifeAlgorithmHydraulic jumpGraphical user interfaceModul <Datentyp>MaizeSign (mathematics)ModemLaptopArtistic renderingPoint cloudServer (computing)Endliche ModelltheorieComputer programmingVirtual realityRandom numberGroup actionSource codeCompilation albumPixelMedical imagingBit rateNetwork topologyCodeLine (geometry)Module (mathematics)Revision controlCopyright infringementEvent horizonMoment (mathematics)Functional (mathematics)BuildingArray data structureWebsiteCircleQuantum stateCommodore VIC-20Maxima and minimaComputer graphics (computer science)Computational visualisticsLink (knot theory)Electric generatorBitUniverse (mathematics)Server (computing)ProgrammschleifeContent (media)TriangleGraphical user interfaceScripting languagePosition operatorLaptopTwo-dimensional spaceProjective planeVirtualizationVector graphicsPresentation of a groupRay tracingFocus (optics)PiPoint cloudSource codePoint (geometry)YouTubeGame theoryMathematicsMobile appMereologyPersonal computerEndliche ModelltheorieGraph coloringTheoryAlgorithmComputer programmingOpen sourceSystem callInstallation artGeometryMappingVisualization (computer graphics)Computer animation
08:42
PixelRootNatural numberClique-widthCircleMedical imagingGraph coloringRandomizationLaptopVirtualizationIntegerNumberFrame problemFunction (mathematics)Loop (music)Moment (mathematics)Range (statistics)Copyright infringementPoint (geometry)Position operatorGraphical user interfaceMessage passingZoom lensServer (computing)Keyboard shortcutOrder of magnitudeMereologyAuthorizationLine (geometry)Module (mathematics)MappingComputer fileBuffer solutionImage resolutionPixelMetadataConnectivity (graph theory)GeometryEvent horizonSocial classCodeSource codeSoftware bugPublic domainDifferent (Kate Ryan album)Repository (publishing)TouchscreenElectronic visual displaySoftware frameworkCloningMathematicsStructural loadComputer animation
16:28
Correlation and dependenceMultiplication signBefehlsprozessorMandelbrot setPower (physics)Well-formed formulaMoment (mathematics)AlgorithmPixel2 (number)NumberZoom lensLoop (music)Pointer (computer programming)Quantum stateInformationData conversionReal-time operating systemMereologyPlotterProcedural programmingMedical imagingNoise (electronics)Functional (mathematics)Commodore VIC-20CircleRadiusPattern languageLine (geometry)Electronic visual displayBitModulformLaptopCollisionDemosceneExclusive orCopyright infringementGraph coloringModule (mathematics)Operator (mathematics)Frame problemServer (computing)Magnetic stripe cardResultantSimplex algorithmFormal languageDifferent (Kate Ryan album)Clique-widthModulo (jargon)Combinational logic
24:14
LaptopMereologyServer (computing)Web 2.0RobotComputer animationMeeting/Interview
24:42
Medical imagingBitService (economics)Web 2.0Server (computing)LaptopCopyright infringementPixelGraphical user interfaceComputer animation
25:46
MathematicsDifferent (Kate Ryan album)1 (number)Moment (mathematics)Revision controlProjective planeQuantum stateMessage passingOpen setWindowBefehlsprozessorPhysical systemGraphics processing unitDifferenz <Mathematik>Bit rateMeeting/Interview
Transcript: English(auto-generated)
00:06
Right, so next up is Martin and Martin is going to talk about a tool called PIRATE for ray tracing and image generation. Martin is a professor of Geoinformatics and Computer Graphics at the Institute of Geomatics Engineering at the University of Applied Sciences
00:26
Northwestern Switzerland. You may remember we had the conference there in Basel last year. So off you go Martin, please start your screen sharing and then
00:41
Excellent. So I'm working. Perfect. Yes. Thank you for the introduction. I just had some technical difficulties Just solve it in time. Perfect. I hope you can see this full screen. Perfect. Yeah, I move this one, too Okay, I'm talking about PIRATE, computer graphics and Jupyter notebooks and the most important part for fun
01:02
And also a little bit for teaching I'm actually teaching Python at the University of Applied Science and Arts, Northwestern, Switzerland. If you attended Euro-Python last year and went to the workshops or the social event that was this building I'm working and today I'm talking about a
01:24
small side project. I started creating a couple years ago and still maintain it And it's basically most for fun to create such graphics we see here My my inspiration or motivation lies back in the in the early personal computers like here the Commodore
01:45
16 or 64 That's actually from the manual of this computer. So it came out with this ugly basic code and you could create Graphics we saw in the previous talks to talk also some some nice graphics
02:02
And problem, of course is always in Python. We don't have something built in to create that I really want something I can create a couple lines of code and and see something that's my basic motivation Of course, you could also create something for game development That's not my focus at the moment
02:22
I was in the game development industry many years ago, but at the moment I'm not creating any games anymore. So I have two other things in mind The first is the server-side graphics generation I want to have a server that can create some some movies for example or animations.
02:43
I also want it for teaching. For example, it's perfect to to explain how loops work if you have some graphics Or of course sorting algorithms, you can display how it works, etc Also, it's nice to create some real-time content for streaming on Twitch, YouTube or now Zoom, whatever
03:07
I will show something like that later on So there are many other modules doing the same thing. So I Don't really go into these just for reference. There are more of these
03:20
For 2D graphics, we have many things. For example, Arcade is more for games, Pexel is more for retro games TV is also for mobile applications, and then there are tons of GUI toolkits where you can create graphics too And of course MoviePie where you can create movies out of still images. The same for 3D graphics
03:43
There are some modules, of course OpenGL Would be the biggest overkill I said I want to create some graphics in a few lines of code. If I would start with OpenGL I would end up with 200 lines of code just to draw a triangle
04:00
That's not really what I want to do at the moment. And of course the GUI toolkits could do the same Also, there are Blender Cinema 4D. You can script using Python And that's all nice, but now I want to reinvent the wheel. So I created Pirate
04:21
It's on GitHub, you can download it. It's a ray tracer and the image generator You need at least Python 3.5 And I had some things in mind. This is what will come in future. It's not yet all working so I want High quality ray tracing in the Jupyter Notebook. I want to use it for teaching computer graphics and ray tracing
04:46
Concepts. I want to use these concepts to visualize geo data using using Python including some things like Octrees, large point clouds, etc. And Also, I have focus on large 3D signal models
05:02
We actually used a modified version of Pirate already to render large 3D city models I'm also not going into that at the moment and Also, we can use OpenSweep map data creating maps out of it If you're interested in it, you can contact me and I can show you how it was done
05:24
And of course the server-side rendering. That's to have fun programming graphics So installation is quite easy. I don't have a conda installation yet
05:41
I only have a pip installation. You can also use conda of course and use pip and it has no dependencies so you just install Pirate, pip install PyRT and It would work without any modules But I highly recommend to have numpy and pillow if you don't have numpy or pillow you can still use it
06:05
But you can't really create images out of it. So you just can create arrays with RGB values So at least I recommend numpy and pillow as minimal installation So let me show you how this works after
06:22
importing some stuff here. I will explain that in an instant live. You can create a so-called RGB image and that's basically just a virtual frame buffer So you have this frame buffer where you can put pixels in it. Internally
06:40
it's just a numpy array or a Python array if you don't have numpy and then You can put the pixels in there. You could also do that manually You could get this array and do your stuff with numpy for example But that's not the primary goal So there are some functions or methods like draw point, draw circle, etc
07:05
And you can then draw your things Internally, it's all done over vectors. So for colors you have a three dimensional vector with RGB values Those are between 0 and 1. It's not between
07:21
0 and 255. So in theory we can have better quality than 8 bit per pixel and Position is a two dimensional vector would be just a coordinate of our virtual frame buffer here And then you just call image frame buffer or just call image into bit notebook
07:44
And if you have pillow installed you will see the image. So I could show this using this presentation, but I prefer to show you that live. So let's switch to Jupyter The source code will be, actually it is already a version there at the moment
08:04
But I will update if I do some changes in the code So it will be located github.com, Martin Kristin, Europyson 2020 This link is also in the room on Discord So let me start it by showing you something
08:23
I had the link here from github of the project You can check this one, you can clone this one for example I did that. It's an alternative way if you want to install it using pip You can install that or you can just go to the source code
08:43
Clone this repository I already did that, I called it pirate So I start, let me move this one I start JupyterLab I could also start a Jupyter notebook, doesn't really matter
09:02
And so far this works I will go not really to full screen We have 125 percent, that's okay So inside there you would see the source code of pirate That's here, so that's also if you want to develop something using pirate
09:25
You have for example maths, here you have the vector3 class And you also see the code here And you can edit the code and you could develop that And at the end you could make a pull request
09:42
And maybe I would allow this one Okay, the most important thing I have at the moment No author dependencies, just nothing It's pure Python So what I do is I just create again this virtual frame buffer
10:00
So I would just import RGB image from the PyRT render And I import the vec2 and vec3 And I use the random module to draw some random points So let me execute that one For some reasons my keyboard wasn't working
10:23
So I create a virtual frame buffer I do it rather small, you can do what you want here I create an RGB image 320 to 240 That's a really small resolution I clear it with black, no not really black But I could do completely black for example
10:41
And for some reasons I messed up my keyboard I have to manually execute that Okay, so what I do now here is I do a for loop in the range of 5000 And I draw my points at the position and color
11:01
And the position is as I said before Just a random integer between zero and the width And also a random integer between zero and the height This minus one of course is because we have one too much From zero to you know that And color is a random value RGB between zero and one each component
11:23
So we have completely random colors So I executed that already And then with image frame buffer I can display that in the Jupyter notebook And hopefully you can see that And see all the points in different colors
11:44
So if you don't see that I could actually increase the point size here And you see more points now with bigger size So I have here some message from Zoom
12:00
That was probably the reason why my keyboard is not working So now it should go again, perfect Okay, so what I do now is I draw some lines I do a range zero to 100 And oh it's still not working but it doesn't matter
12:22
And you see a live update now You see the lines are drawn over this image before This is working because I created this frame buffer before And I can use it again So if I create one frame buffer it's reusable
12:41
And you can draw your things inside Of course if I want to create more frame buffer I can do that too And then I have to give each frame buffer a name So I don't draw to this one I draw to another one I will show that in an instant I also have to set update And I can specify recommended frames per second
13:05
The number of frames per second You see here I have some debug output We see it was almost a 30 It's not possible to have this exactly for obvious reasons By the way this doesn't work on Firefox at the moment
13:21
That's a known bug I will fix that at one point at the moment I recommend using Chrome So I can also load images I have some images in this folder here A world map I can show that quickly here in data
13:41
I have a world map And that's just a JPEG of the world from natural earth It's a public domain dataset So I don't even have to say from where it is But I just said it So no problem And now you see I create a frame buffer with a name World And from now on if I want to update that
14:00
I have to tell okay update the frame buffer world And I do just for fun I draw a circle here And we see in the middle 300, 150 There is a circle I could also create some random circles 100
14:20
So we see this is actually working And we see these random circles So don't worry I will not create a COVID map We saw enough of those So I decided to download earthquake data from the USGS So that's actually a GeoJSON file on the server
14:43
And here it's explained what it is There are tons of metadata inside And the most important part is the geometry of course There you have the longitude, latitude and depth But we are in Tuesday So I don't really care about the elevation of this event
15:02
And the magnitude of the earthquake So if it's earthquake 7 That would be not very nice to see So let's see what's That's actually live I didn't check it this morning So I don't really know what will happen If there was a bigger earthquake or not I will download your old earthquake data from the last two and a half weeks
15:26
I download that as a file called earthquakes GeoJSON So let me do that So there should be a file now Earthquakes GeoJSON So that worked And I load it as JSON
15:41
This is basic Python And then I load the image again And I go through this JSON file And I get all the coordinates out of it And I convert the longitude and latitude to pixels That's done here with using the coordinate And converting it to simple pixels And arrange 0 to the width and height
16:04
And then if the magnitude is bigger than 4.5 I draw a circle I could reduce that to 2 But I don't really want too many earthquakes Because there are many small earthquakes happening And at the end I display this image
16:21
So let's just do that And we see here are the earthquakes of the past two and a half weeks Of course it's not really a nice graphic You can't really use the mouse pointer And get more information like this It's really just a plot
16:40
This one I could do server side with pirate And do an update every hour for example That would already work In the last talk we heard a lot about procedural images This is also one of my favorite topics I will not explain anymore what's noise
17:02
We heard that in the last session So let me just do a basic introduction to procedural graphics Again I create an RGB image The size is 256 by 256 And then I create some stripes
17:23
Let me execute this first Then I can explain You see the result is this one And such stripes can be easily done using a modulo function I have a modulo function in pirate inside You all know with python you can also do modulo using this percent operator
17:43
I did this because this modulo function uses floats Of course python supports floats too But you can easily mess up if you don't really use floats etc And I did it for readability So if you prefer this syntax it's fine I prefer this one
18:01
I can read this better I don't know why I just don't like this syntax And this one is better portable to other languages such as C++ So this modulo and this 0.5 Let me do a 0.25 just to show you the difference You see this one influences the width of these lines
18:26
And if I make 0.5 I have even black and white and black and white I could of course do other colors I just set this to black or to white like this So I can do the other thing
18:41
I can use y instead of x And then it would just look like that So you have horizontal lines here And now the big question is how can we combine that If I want to make a checkerboard 8x8 So like this and this No it's not and It's xor
19:00
So we do the same And I do the modulo operation here for the vertical and horizontal lines And in between is an xor So if I execute that you see it's really a checkerboard Starting with white, black, white, black and so on And this already starts xor
19:22
This is a perfect way to explain xor If I do By the way I never understand why Python doesn't have this one But it doesn't matter I do an or and you see if I do a regular or You see I get this pattern because it's this or this So it's that
19:41
If I make an and for example You see the solution is of course just the combination of the two So where the two lines collide So that's the xor And we see here we have checkerboard So that's an easy way to understand xor in a graphical way
20:05
So I'm running out of time as expected Doesn't matter I create quickly a Mandelbrot image The formula for the Mandelbrot you can find anywhere For example on Wikipedia or whatever And it's a really simple algorithm
20:22
You just have to implement this part basically And you have to Here this one means that I convert the pixels to a number So zero zero is in the middle And it goes from zero to one and so on
20:40
So that's just a conversion And then I put the result of this while loop to the image So we see nothing happened because this is just the function Now I time it and I create this Mandelbrot image And we see this took 1.5 seconds to create and display it
21:09
So that's okay I remember my first Mandelbrot was a little bit smaller than this Took about three to four minutes on a Commodore Amiga end of the 80s
21:22
So this is still too slow So we could use number for example to speed it up So I just use number make chit Do the same here We see of course this while loop will not be able to break down But of course we can still try what happens
21:41
So I execute that one And then let's time it We remember before it took 1.5 seconds And now it takes 686 milliseconds So I execute it again We see now it only takes
22:01
We only have 22 The problem is we have to compile it one time And the second time we execute it, it's really fast So we are at 21 seconds And so we can also do noise Simplex noise for example I will go through this a little bit fast We saw that in the previous talk You can really create many nice things using these
22:27
Simplex noise or noise functions like the Perlin noise But one thing I want to mention before I stop for the questions You can also create movies using MoviePine Let me just quickly explain that
22:42
There is a function called for example make frame Just a function you can name it like you want But in there there is a time And then you draw something And time should be a function of that So I just draw a circle and I use time to use the radius of the circle
23:02
So I do that And now in background it creates this GIF And if I display that we see this animation So I can even create animations in the Jupyter notebook quite simple But one last thing I do this quickly is the ray tracing part You can create scenes, you can create some things
23:24
And now I start rendering this This is really pure Python It takes about 10 seconds And then it should be finished Maybe because Zoom runs it takes my CPU power I see my CPU runs at 30% at the moment
23:40
So it took me 12 seconds to create this image here And this is really pure Python We also tried this to accelerate this And you can actually we used FireGL in background And we had this in less than a We can display this in real time actually using these things
24:03
But I want to keep Pirate at the moment pure Python Okay so thank you very much And I'm ready for questions if the time still is sufficient So I don't see any questions in the Q&A of the webinar
24:21
But I have a question Is it possible to run this as part of a web server as well? Because you just focus on the notebooks But let's say you want to dynamically generate a GIF Or maybe you want to write a bot for Discord let's say And you want to produce GIFs that are animated Would that work?
24:41
Yes of course You just have to skip this one This is of course this only runs in Jupyter But for example if you write something Actually you have what you get out from Pirate If you have NumPy installed you get a NumPy array, RGB array So let me show that quickly
25:02
This is not the best example but I do it anyways So I do image data Actually I should execute that quickly because And you see this one is just a NumPy array filled with RGB values
25:20
It's a Uint8 so it's 8 bit per pixel Now it's all black because most things are black at the corner So you can just save You can save that and this is really doable on a server You can create a service You don't need a GUI for anything This is just for let me say debug purposes to display this image in Jupyter notebook
25:43
So you could create anything on a web server Okay excellent thank you Then we have two more questions I think we can take at least one So first question is do you need a GPU for ray tracing? No In the current version which is on GitHub we don't have that
26:02
We did this in a project We used open from AMD if there is some some toolkit called open fire or something And they have a ray tracing engine on GPU And we use that
26:22
But the problem is we have then 1000 dependencies And we have for example in Windows we have two DLLs on Mac For each system we have different ones And it's really really messy So we never released that in GitHub And for now it's just pure python and we don't use any dependencies
26:42
So at the moment it's CPU only this version Okay thank you very much So there's one more question but I would like to defer that to the talk channel And so let's have your applause Thank you
27:03
So thank you very much Now we're going to have a message from our sponsor