Rendering complex 3D-Geodata using pyRT

Video in TIB AV-Portal: Rendering complex 3D-Geodata using pyRT

Formal Metadata

Rendering complex 3D-Geodata using pyRT
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Rendering complex 3D-Geodata using pyRT [EuroPython 2017 - Talk - 2017-07-12 - Arengo] [Rimini, Italy] PyRT (pronounced "pirate") is a rather new open source project creating a ray tracer in pure Python and some optional CPU/GPU acceleration using bindings. Ray tracing is a technique for generating an image by tracing the path of light. PyRT was created to render large 3D City models. In this talk, the possibilities and experiences of ray tracing in Python using pyRT are shown. pyRT also runs in the Jupyter Notebook. Rendering complex 3D-Geodata, such as 3D-City models with an extremely high polygon count and a vast amount of textures at interactive framerates is still a very challenging task, especially on mobile devices. This talk presents an approach for processing, caching and serving massive geospatial data in a cloud-based environment for large scale, out-of-core, highly scalable 3D scene rendering in a web-based solution. PyRT is used for rendering large amounts of geospatial data. The approach for processing, rendering and caching 3D-City Models is shown.
Scaling (geometry) Copyright infringement Forest Software Metropolitan area network Demoscene
Digital photography Goodness of fit Graphical user interface Theory of relativity Globale Beleuchtung Computer-generated imagery Globale Beleuchtung Directed set Right angle Local ring Resultant
Point (geometry) Workload Electronic data processing Digital photography Open source Revision control Point cloud Process (computing) Mereology Declarative programming Directed graph
Default (computer science) Building Standard deviation Texture mapping Mapping Demo (music) Civil engineering Shape (magazine) Mathematical model Web browser Demoscene Geometry Right angle Object (grammar)
Geometry Building Dew point Domain name
Geometry Building Bit Fingerprint Directed graph
Ellipsoid Mapping Code Multiplication sign Projective plane Geometry Bit rate Personal digital assistant Different (Kate Ryan album) Energy level Energy level Library (computing) Dean number Physical system Library (computing)
Default (computer science) Mapping Tesselation File format Texture mapping Computer-generated imagery Translation (relic) Limit (category theory) Mathematical model Type theory Geometry Type theory Vector space Different (Kate Ryan album) Vector space Process (computing) Pixel Library (computing) Geometry Library (computing)
Slide rule Building Computer file Multiplication sign Mobile Web Computer-generated imagery Tesselation Image processing Set (mathematics) Parameter (computer programming) Open set Web browser Mereology Demoscene Mathematical model Usability Read-only memory Semiconductor memory Artistic rendering Energy level Process (computing) Physical system Point cloud Mobile Web Zoom lens Mapping Building Parallel port Usability Greatest element Demoscene Mathematical model Process (computing) Google Maps Artistic rendering Quad-Baum Point cloud Right angle Energy level Quicksort
Multiplication Mapping Computer-generated imagery Zoom lens Interactive television Selectivity (electronic) Mereology Demoscene
Principal ideal Group action Building Pixel Computer file Mereology Graph coloring Medical imaging Regular graph Artistic rendering Aerodynamics Acoustic shadow Normal (geometry) Position operator Physical system Modal logic Matching (graph theory) Mapping Information Forcing (mathematics) Ray tracing Projective plane Interior (topology) Client (computing) Bit System call Type theory Normal (geometry) Simulation
INTEGRAL Code Multiplication sign Computer-generated imagery Disintegration Set (mathematics) Mass Inversion (music) Mathematical model Mathematical model Computer programming Medical imaging Graphical user interface Different (Kate Ryan album) Vertex (graph theory) Normal (geometry) Triangle Point cloud Module (mathematics) Mathematical model Mapping Copyright infringement File format Demoscene Mathematics Artistic rendering Normal (geometry) Point cloud Object (grammar) Quicksort Computer forensics Laptop Geometry
Laptop Pixel Quantum state INTEGRAL Graph (mathematics) Texture mapping Multiplication sign Disintegration Computer-generated imagery Demoscene Medical imaging Geometry Crash (computing) Array data structure Bit rate Object (grammar) Acoustic shadow Pixel Ray tracing Code Hierarchy Right angle Object (grammar) Volume Resultant Laptop Library (computing) Geometry
Quantum state Quantum state Virtual machine Maxima and minima FLOPS Disk read-and-write head Flow separation Power (physics) Band matrix Moore's law Virtual memory Bit rate Read-only memory Semiconductor memory Kernel (computing) Befehlsprozessor Band matrix Forest Right angle Force Graphics processing unit
Array data structure Arithmetic mean Globale Beleuchtung Bit Object (grammar) Mathematical model Mathematical model Computer programming Resultant Writing Number
Real-time operating system Mathematical model Number Mathematical model Geometry Inclusion map Plane (geometry) Different (Kate Ryan album) Video game Universe (mathematics) Software testing Gamma function Directed graph
Mapping Multiplication sign Musical ensemble Metric system Mathematical model Demoscene Frame problem Mathematical model
Medical imaging Building Satellite Mapping Computer-generated imagery Website Student's t-test Electronic visual display
Frame problem Video projector Demo (music) View (database) String (computer science) Computer file Demo (music) File format Gamma function Maß <Mathematik> Supercomputer
Point (geometry) Surface Service (economics) Building View (database) Venn diagram Direction (geometry) Structural load Direction (geometry) View (database) Computer-generated imagery Mereology Connected space Mathematics Internetworking Right angle
Surface Service (economics) Pixel Mapping Key (cryptography) View (database) Direction (geometry) Computer-generated imagery Moment (mathematics) Data storage device Graph coloring 10 (number) Mathematics Medical imaging Process (computing) Normed vector space Point cloud Quicksort Quantum Normal (geometry)
Building Mapping Demo (music) Internetworking Directed set Online help Connected space
Overlay-Netz Surface Service (economics) Building Demon Mapping View (database) Online help Direction (geometry) Computer-generated imagery Maxima and minima Shape (magazine) Expert system Power (physics) Mathematics Spring (hydrology) Software development kit Dew point Summierbarkeit Gamma function
Intel Asynchronous Transfer Mode Quantum state Projective plane Ray tracing Code Design by contract Bit Open set Mathematical model Flow separation Medical imaging Particle system Kernel (computing) Different (Kate Ryan album) Cylinder (geometry) Kernel (computing) Right angle Asynchronous Transfer Mode Graphics processing unit
Point (geometry) Metre Functional (mathematics) State of matter Multiplication sign Range (statistics) Execution unit Sheaf (mathematics) Design by contract Real-time operating system Open set Mereology Machine vision Power (physics) Software cracking Position operator Form (programming) Physical system Mapping Moment (mathematics) Computer simulation Bit 3 (number) Measurement Flow separation Microprocessor Doubling the cube Personal digital assistant Order (biology) Normal (geometry) Pattern language Object (grammar) Quicksort Resultant Geometry
Revision control Decision theory Artistic rendering Projective plane
thank you very much for the instruction the yes and talking about a pirate pronounced correctly priority pirate hat on but
before cost about what I want to and forest I want to have a way to render a man scenes from indoor to outdoor and the mean from and plan that really bad at scale to that's the 1st challenge or at least I tried to the
2nd challenge I want to have a good quality and good quality means and you want to use global relations so I really don't want them to have the result we see in the left I want to have to result in the right so if you paint Walgreen or at the Newseum the floor it's also little bit greenish and reddish so of global illumination so we really and follows the path of flat so let me just
show you my previous work what did do with the past 12 12 to 17 years and I created a
virtual started in 2005 you probably know Loris for all the world you build knowledge is and I created my own it's called open workload and was originally written in C + + and then in 2011 and with the advent of lecture years we moved to just strict 1 side of the Python conferences Sroda tomatoes at what it's really jealous declarative and you can do some cool stuff like alan in much geodata you can have all the photos and you can even have point clouds and added large-scale point clouds you had and millions of points billions of points on the road to blow up so the data processing part was also written in C + + and was moved to Python so there is Python positive can put away all the tomatoes against it the exact and
let me show you how it looked like and here we have a dataset from Switzerland and come about a sink us we care about the data and can fly around new don't
see really buildings here you see the elevation model and or defaults Saul and 2030 animosities this started to add buildings but the big problem our buildings these there you get buildings of the whole planet if you are not going to or if you don't have a budget of a few billion dollars so 1 solution they came up with this why don't we
use OpenStreetMap data so maybe you know OpenStreetMap it's a 2 D map and you know many people do mapping was crowdsourcing so you can really and add your own buildings and there are some people who actually um and at some tax off the roof shape for example or of building height etc. so what if you take this data and create 3 D model of the whole world so that's what we did to see them on the top left the scene in Switzerland where people didn't really have the building height so we estimated standard building height there on the right side top his New York it's much better there from someone of some people really dates at the re shapes of the buildings and that's really inside the the OpenStreetMap that doesn't and the bottom right to the heart we wanted to add some because the buildings of interest has some some texture object in with the buildings of really interest very really have takes a object and can replace that from the OpenStreetMap datasets I'm on the bottom left you see the Forbidden City in Beijing and its see the of the of typical roofs are also not there yet
so let me show you rarer briefly how it worked we all use the OpenStreetMap data are strong later and
added some buildings years have been of interest pyramint the material for the especially consume around then you can go to another city async more we froze no OK you just
so then let's go to the and you see it the loading speed the pocket floats some buildings appearance assimilated with a bit to bond but and with their ultimate but it's a common bond ways you have missed this on a sink the normal
phone so sweet she and then the buildings
of interest or it'll you see the rate is not
complete because this is missing in the OpenStreetMap dataset OK that's what it
looked like and so how do we do it
um it's quite easy to use some projection that don't want to go into details here also was Python there's a great library code approach for very many many and projection systems cartographic project assistants are included and everything is of course what Freebase this the same principle like to Woody maps where consuming and some out the only difference is that you look at the world you go from side so you have a different level of details at the same time
and you also have to consider different type types so you have to classical default the imagery the kind of to vector data you can have 2 Woody elevation tile white Woody because it just is a height map basically and this height map is converted to a 3 D model this yellow shade receded OK so we also have some special
geometry tires so Miss special translations to have a better quality and become and remain in use and Uchida library and with the Python binding to access roster data including the formation OK that was the previous work
so yeah I was not really happy with it some this is all the rendering speed is not really that great and another problem is the ease of navigation is not that great maybe all of you know Google Earth most people look at that 1 time maybe 2 times and then that end up using Google Maps that's the truth unfortunately so and another problem is under the support for mobile devices like that right where she runs on mobile devices but you really need the latest this latest and most expensive to wise to run such a scene and of course the most important argument i want more Python so we came up with some
ideas why don't we just take the on the system of to the mapping system and render it from the side so you actually see this really um building but it's just a totally up so it's the same liketo so you can have sticky model from for example Rimini in 6 level of details and if you do with 6 level of detail as you probably have tool to sort of 3 buildings part time so that's not much if you have to to swing buildings that have become really render that extremely fast so all the good thing is you never need the holy city model in memory even you your ended up because you only render 1 times the time and this means the rendering is really fast and the best part is I can run all these processes in the cloud so this can be really gone with in parallel by request or I can the renderer so how it looked like so have assume level and I didn't make too many just for for this 1 slide so you understand the principle and I say OK ensue level and I render it in the the best quality is an offline right and then was Image Processing I create the remaining level of the 1st so let me show you an example we use of open data set of the city of Rotterdam um agree with that because of that consists of city chairman files of Iran's reaching me and has some pictures of about 500 GB so here's a browser we
enter but not the entity where
undersea multi 1 2nd later you come interact in this scene and the good thing is we can do some more stuff like interaction so we can select some parts consuming zoom out so now we wonder how did we do with this selection in 2 the map that's also quite easy reuse of our
old principal from the 19 nineties called achieve offer achieve offer is a group of image types 1st you have to color information just color of the buildings the 2nd is a normal map where you store enormous each pixel the search is the classification type of map it's a caller ID map so for every building or rebuilding group you have the color so this discolored can get the idea DID back and then the force didn't really put a picture here because it makes no sense is the best match for every pixel you can get the best value and this this this file you can you can we calculate the exact sweetie position in this seen so I can even measure it within this system it so this is the principle now the hard part was How do we render it for the 1st thing we did we use the random and it's a best friend the package of the world then we used call for a
great package here see on the on the left is seen in the previous ray-tracing you see that project resulted that good but you see the picture quality is much better and even have shadow there on the on the right side on the left side This is a classic question based in your so we use the for about the problem of frailest that we did not have a good support of the best MAP and of norm map because we want to have a high precision depth map with 60 at least 50 64 thousand bits per pixel so the only solution of course most trait
of all the and that's the pirate is coming from character is a right tracer which uses python C 2 . 5 must be stricken 5 can't be less than 3 . 5 11 book about that in an instant and the primary goal is we want to render high-quality images we want to support several different and Brendan techniques and different lighting models and of course you don't have to pick the integration so they can render an images in the tube we do not look but the most important as I mentioned before is we want to have prices precise maps normal maps and object ID maps and you want to crate that in the same brand respect so we don't have to render for images so you just run the 1 time and they get all this information commercial and open-source refers to not have to support and also the important thing is we don't want to have a forensic we'd like Blender we want to run this in the cloud and you want to have it really is a set of command line so let me show you simple
Hello World and it's highly packaged into different submodules so we have a mass tumor material camera and so on modeling of render modular which can be extracted so you can after all renderer the the 1st to create a camera you seeing you abstract no sort geometry to your scene specified material and then to create render and render it so now you may think why do we do that this complicated the actually write code to to render a scene so is this is quite easy to support different formats and to be at the end you have a Python program which describes the this is how the actual beach
notebook and the crash that looks like so you can really of Europe and geometry there you can render it and you see it the 1 thing I didn't mention at this it doesn't really is pure price and its use does not depend on any library however for for this Jupiter integration we check if pill is available to create an image and the check of course if Jupiter is available so it can and are correct result in HTML so let me quickly show the
principle of ray-tracing it's really a very simple principle and we have a light source and from this light source of race and maybe this race the hidden objects and maybe this object of reflects this writers and maybe have very lucky use race the the camera so maybe we should 1 trillion race and 20 Saudi has arrays hit the camera so the is quite expensive so um and then some people came up with the idea why not to undo the backwards we start from the camera we should some race we take an object and then you see if there is some a light or maybe it's prudent so we have shadow this this principle we only need the the minimum rate 1 rate pair resulting pixel so if you have full HD you only need 2 million rights and coming back to that you all know how fast Python this so OK so we have some more fuel the features I've already told of a few things about that and I'm not going to date has put a C. the time is running short but
now the speed to really accelerate um pirate the head master this some of last year I'm actually finish this year and it use the rating raise API right raise a light is a ray tracing coroner well you can do and 1 saying you should praise and check if it hits something it's really embarrassing and it uses internally open Seattle some to support different GP was and use and for our experiments we use and then we get she forests 1080 TI some it's it has 8 GB of memory map of memory bandwidth of 320 gigabytes per 2nd a really nice and theoretical um compute power of 11 Teraflops I'm not going to give you can look it up if you bought the 11 Teraflops machine 20 years ago you know you need several billions and billions of dollars to do that so
euros does not have this much OK so let me show the result so if you the Lewis may make different with native Python and the on on on my machine we came up with 12 thousand race per 2nd and if we use we still use python of course and the cheap you really have almost 90 million raise per 2nd so it's really big increase of compute power
so let's look a little bit but that um lighting models along in some very nice and and easy to program and lighting model is to ambient occlusion model it's it's results if you compare this is no writing of a stochastic found it really looks much better and and but it's a it's a fake actually it's it's not a real global illumination but it just looks good basically and it's quite easy to implement now of course that would 0 it's really easy to just shoot array and then you see how many arrays if you should it again it on an object so you get an occlusion and the number and if many many race hit another object and stock and if for example you know radius it almost no raise it so it's like the so it's a really simple to implement and solution so I
mean they're masterpieces of Marcos very implemented that and the result of looks not too bad so if you have if you you see the number
here you can check you can test different number of rise again if we use a number of
200 already looks quite well you can use a filter to increase quality
and if you compare this with the regular plane from a city model for example looks in a sink it looks much nicer and
if you go if assuming into a city model to see if you compare it with the open phone used in real-time graphics mostly OK you can also use and inclusion in real-time graphics if you look at computer games but it looks really quite quite cold or
another scene and recently created some looks really much better if you invest some time musical lighting model so um we come back to the
occlusion maps so we want to have a map at the end so this movie shows how to create such a metric can really interact with this this is kind and pre-calculated could also precalculate some movement if you want but you need really lots of frames and and do were really interactive and you of of this of this city so in this year we had another master
users to come back to the OpenStreetMap datasets of we wanted to have a global way to render old buildings of the world the OpenStreetMap data so the solution the student came up with was and why don't we just take the OpenStreetMap to the map and just put the buildings on it 3 so from the site of the fates to audience really combined in but it looks it looks a lot about so we get this with
in New York City here is the in some ways was a subword image from map books combined with the this this from Brenda technique and let me show you quickly a lifetime if I have time at
the time of of 10 minutes for it so I was too fast already so we have to wait for the projector which seems to be slow to
their no problem at all the from
director here um 1
2nd but the major demo 1 it's actually New York City you
see this is just in Manhattan basically a part of it and you consuming now I hope I have and the connection this great to have a conference with a slow internet connection was at this point because if it's slow you we see something in it so that's the end up in the background you see the normal opens with update them and in the front you see the buildings render so that's about how it looks like and then you can say OK I don't want to see here is a right so see how it quickly loads and the slow connection and you can also change the
view direction here we only made
for from each side of course you could have more if you want it's also a matter of cost of of cloud storage basically and here you can also see the
enormous you can have a color
map so here and then this just costs and to their father however that not all countries see anything because it makes no sense in warm and float stored as they make each per pixel so use so I'm in this disguise sort key to base that per pixel and stored at us as an image so why an image so you can download it from the cloud and process it again and and converted to the moment of my
so that's basically the demo I want to show you see isn't it right so it's really it's really the to the if it if you look here is that to woody map from from pulp this this standard OpenStreetMap so it's it's really OpenStreetMap lots little OK another internet connection is that but it's a 2 D map from top and all the help we we at least 3 D building so it's really a fake but it looks really like like 3 D and they can do that and that's the nice thing on the on the whole planet so let me show a 2nd
demonstration of a city which doesn't have
nice shapes the city of Basel where I live here to attend just so and it's truly map overlay there are some some buildings for example this Roche power here is sum someone really did the work and added to OpenStreetMap now we just have to wait if everyone if every person in the city creates 1 building and then we have a nice map In the city this is how it works and is really fast for the OK it's a commission just at a Python
again but I think it's it's a good way so let me come slowly to get and so we can have Q. and a
and notice projection would be slow so 1 2 3 4 5
that I don't know but it OK let me start as the conclusion already while we wait for the image we
still pirate quality greater we saw pirate an open-source right for a which has been written completely pricing with some additions like and of the the Reagan raise API which is open CO and C + + um but because it's paid licensing to call this this really of at least from the Python particle this readable and understandable and to be already support some different lighting um models so I didn't modes and I didn't really I'm sure you all of them yet some of missed due to exploration really have a fast solution and to be already used parrot in several projects and 1 was this recent masterpieces was the OpenStreetMap data and we also have another project ongoing which I can't show what the moment because of some contracts but there are and parrot is used to form and outlook We are currently on refactoring the mouse a a bit to put it on the top soon and and in future we also want to use more ray-tracing kernels for example in Intel Embree gamma ray tracing current which CQ based which doesn't use the chief you but this is still very fast because it uses all the more and things of of a CPO and um cylinder be another multisystemic true at more terror and and all of those things to do with ray-tracing
so thank you very much for your attention thank you thank you thank you very much mountain before this movement of the talk in the mole of vision after this thing some time for questions so you free the thanks when we see uh it you use to was very my mind that we're you mean how the connect system to be used in the summer on the walls and it's a erupted about the CPU chip you part is completely written in C C + + and then we have findings and and call these these functions however this is just another optional away so the basic phrase you can you can have it you don't need a cheap you just have to wait and to get the same result the that's how well various sort and who that this you or you can usually so you know what he's precision of the 1st section of the beam at height these days you made your contract following light etc. away like some light would would new this the spot here you can see the genomic very very tiny place we want to simulate sunlight III set in each year in acquisition of or you know they don't care for those units know yes we have enough precision there are some tricks there are several tricks actually to increase precision 1 very simply the trick is to have for example several of them you frustum st and you decide which you Frost may use depending how far the object is power this this solution has problems there is a very elegant solution by using virtual cameras that have offset in the camera and you just said is also 0 0 0 and offset order geometry in the time and then you can have freed from from far a 20 year can have on the whole planet and consume tool to come to a house even inside house without without losing position because you always have floating point position of the in in this in this case because and in the in the vertical for example you are limited to floating points because it's it's the way it's cheap you have state at the moment to become but that was all common chickens indirect race on reuse double precision for everything and you can have cost a little bit better position especially for the for Deep Map this is very important in because we want to measure in the south millimeter and range so we have some use cases to 2 them for example measure some cracks in restraining Wallace or patterns and there we really need to solve many millimetre and there we can create normal maps and depth maps with this position hello than would fall and I 1 to consider some prior to use real-time operating form but what if it was a noticeable quality in a source of everything and that's an option and open
for that of course and but in the 1st
version the we don't want to use that just a decision to to advance project more quickly but in the future we can we can have a chicken out it's it's highly and abstract the renderers is really come this abstract that you can write whatever you want and and it should work the then things again but and