AAA Games with Unreal Engine 4 and Python
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 160 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/33792 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201733 / 160
10
14
17
19
21
32
37
39
40
41
43
46
54
57
70
73
85
89
92
95
98
99
102
103
108
113
114
115
119
121
122
130
135
136
141
142
143
146
149
153
157
158
00:00
Game theoryIntelDivision (mathematics)Level (video gaming)Software frameworkVideo gameComputer animationLecture/ConferenceJSONXML
00:54
Game theoryText editorComputer programmingInstallation artSeries (mathematics)Game theoryComputer programmingAcoustic shadowRepresentation (politics)WordSource codeElectronic mailing listComputer simulationOpen sourceBinary codeRepository (publishing)Text editorWebsiteLogicWater vaporCasting (performing arts)FamilyLetterpress printingDefault (computer science)Programmer (hardware)Power (physics)Table (information)Wave packetOrder (biology)Tablet computerComa BerenicesSoftwareReal numberComputer animation
03:39
Scripting languageObject (grammar)Descriptive statisticsLinear algebraOrder (biology)Transformation (genetics)Event horizonSound effectProgrammer (hardware)Form (programming)CASE <Informatik>Keyboard shortcutSpacetimeScaling (geometry)Position operatorWordGame theoryHydraulic jumpLecture/Conference
05:04
Object (grammar)Euclidean vectorEvent horizonPattern languageCodeSelf-organizationData managementError messageGame theoryAutomationFunctional (mathematics)Execution unitSoftware testingDisintegrationVideo game consoleFunction (mathematics)2 (number)Module (mathematics)Event horizonVirtual machineVideo gameUnit testingVideo game consoleProjective planeCodeMereologyDifferent (Kate Ryan album)Data managementSoftware testingSoftware developerLibrary (computing)Materialization (paranormal)Pattern languageCodeText editorSocial classConnectivity (graph theory)Self-organizationFunctional (mathematics)Programming paradigm1 (number)Game theorySkeleton (computer programming)Figurate numberCharacteristic polynomialVisualization (computer graphics)Multiplication signGoodness of fitArtificial neural networkGame controllerVariable (mathematics)Electronic mailing listOnline chatDirection (geometry)Programmer (hardware)Level (video gaming)WindowLogicTask (computing)WritingFreewareOpen sourceReal-time operating systemSoftwareComputer programmingAdditionShader <Informatik>Formal languageCartesian coordinate systemObject (grammar)AuthorizationScripting languageRevision controlPower (physics)Crash (computing)Error messageExpected valueGraphical user interfaceInsertion lossBroadcasting (networking)MathematicsClient (computing)Endliche ModelltheorieCuboidGreatest elementSound effectCycle (graph theory)Set (mathematics)Beta functionCoroutineFacebookCategory of beingSummierbarkeitUsabilityWater vaporCondition numberSparse matrixEinbettung <Mathematik>RootComputer configurationGroup actionSubsetBoolean algebraAreaPressureMaizeTotal S.A.Noise (electronics)Dimensional analysisParticle systemDigital photographyPoint (geometry)Sheaf (mathematics)MassProgram flowchartComputer animation
15:25
Texture mappingObject (grammar)Function (mathematics)Physical systemCache (computing)Field (computer science)Social classReflection (mathematics)StatisticsConvex hullCovering spaceReal numberGraphical user interfaceSocial classFunction (mathematics)outputFigurate numberLine (geometry)VaporHacker (term)Object (grammar)Functional (mathematics)Category of beingExtension (kinesiology)QuicksortForcing (mathematics)Event horizonMappingSoftware developerPressureVirtual machineDivisorDecision theoryStandard deviationLevel (video gaming)Sparse matrixPhysical systemState observerData structureComputer file1 (number)MereologyGame theoryReflection (mathematics)Formal languageSpeicherbereinigungMultiplication signResultantHoaxString (computer science)Equivalence relationCASE <Informatik>Text editorInstance (computer science)Slide ruleSystem callCrash (computing)Scripting languageTheory of relativityHash functionData dictionaryWritingPlug-in (computing)JSONXMLUML
21:41
BuildingGraph (mathematics)Vertex (graph theory)Text editorModule (mathematics)Electric currentCrash (computing)Reflection (mathematics)Physical systemExecution unitSoftware testingVideo game consoleSuite (music)Euclidean vectorExponentiationSoftware testingAbstractionQuicksortSheaf (mathematics)Multiplication signSource codeCASE <Informatik>WindowVirtual machinePhysical systemSoftware developerIncidence algebraInteractive televisionVideo game consoleInterpreter (computing)Pattern languageForm (programming)ExpressionType theoryGroup actionResultantSparse matrixLevel (video gaming)Reflection (mathematics)Crash (computing)Mathematical optimizationError messageState of matterSocial classBlock (periodic table)Formal languagePoint (geometry)LogicUnit testing1 (number)Finite-state machineConnectivity (graph theory)Text editorFunction (mathematics)DatabaseRollback (data management)Shader <Informatik>Data structureOperator (mathematics)Standard errorLine (geometry)Standard deviationFreewareCodeMonster groupGraph (mathematics)Scripting languageArrow of timeImplementationGraph (mathematics)
27:31
Pattern languageProxy serverSocial classIterationFlow separationRun time (program lifecycle phase)CodeWrapper (data mining)System callLatent heatObject (grammar)Social classPattern languageProxy serverCrash (computing)Software testingEvent horizonError messageGame controllerMoment (mathematics)Affine spaceLogicGame theoryDivisorView (database)Point (geometry)2 (number)Right angleReal numberSoftware developerReflection (mathematics)Virtual machineGreatest elementEndliche ModelltheorieAnalytic setPositional notationSpeicherbereinigungIdeal (ethics)Cartesian coordinate systemWeb 2.0Execution unitSet (mathematics)Sparse matrixProjective planeText editorModule (mathematics)Multiplication signServer (computing)Scripting languageComputer fileBytecodeIterationMereologyVirtualizationAttribute grammarTable (information)BuildingFlow separationLecture/Conference
31:22
Function (mathematics)Social classBuildingFormal languageType theoryComputer programmingMeta elementInstance (computer science)MetaprogrammierungCrash (computing)Revision controlDifferent (Kate Ryan album)Mobile WebCovering spaceCoding theoryThread (computing)Tablet computerGame theoryDisintegrationSoftware engineeringDemo (music)Type theoryFunctional (mathematics)Social classPhysical systemObject (grammar)Pie chartProblemorientierte ProgrammierspracheGame theoryInheritance (object-oriented programming)Graph (mathematics)String (computer science)Projective planeGoodness of fit1 (number)ResultantLaptopSoftware developerWindowBlock (periodic table)Hardware description languageCodeImplementationCuboidBinary codeWebsitePreprocessorInterface (computing)Software testingTablet computerINTEGRALThread (computing)Programming paradigmTask (computing)MereologyInformationLogicHoaxGraphical user interfaceScripting languageProxy serverDifferent (Kate Ryan album)Instance (computer science)TwitteroutputDecision theoryAnalytic continuationForcing (mathematics)Real numberInductive reasoningPresentation of a groupCrash (computing)Water vaporComputer programmingRevision controlVideo cardEndliche ModelltheorieImage resolutionDependent and independent variablesGreatest elementOpen setUser interfaceDefault (computer science)Process (computing)Physical lawSign (mathematics)Principle of maximum entropySource codeJSONComputer animation
37:53
Lecture/Conference
39:08
Interior (topology)Pattern recognitionOpen setNeuroinformatikDemosceneTemplate (C++)Standard deviationDistribution (mathematics)Level (video gaming)Phase transitionObject (grammar)QuicksortSpacetimeMachine visionBitSource code
40:05
Electronic mailing listPiDemosceneEndliche ModelltheorieCode2 (number)Social classField (computer science)DivisorRight angleVector spaceGreatest elementAnnihilator (ring theory)WebsiteMultiplication signTable (information)Message passingSpring (hydrology)Dynamical systemMathematicsFigurate numberGame theoryOrder (biology)CASE <Informatik>Connectivity (graph theory)Power (physics)Object (grammar)Computer fileSoftware developerCoefficient of determinationRevision controlVideo game consoleMotion captureTouchscreenSemiconductor memoryTexture mappingAreaGraphics processing unitGraph coloringComputer animationSource code
43:36
Game theoryConnectivity (graph theory)Disk read-and-write headStatuteObject (grammar)Right angleDemosceneRoboticsTouchscreenPiPosition operatorComputer animation
44:25
Convex hullDisk read-and-write headProxy serverPolygon meshObject (grammar)Field (computer science)TouchscreenConnectivity (graph theory)Social classRight angleStapeldateiGroup actionExecution unitIntegrated development environmentData miningMultiplication signVertex (graph theory)Cartesian coordinate systemSource codeComputer animation
46:01
Geometric primitiveSocial classProxy serverDisk read-and-write headSystem callTexture mappingEndliche ModelltheorieDefault (computer science)DivisorDemosceneHypermediaTouchscreenElectronic visual displayRectangleFrequencyVideo gameIndependence (probability theory)Source code
47:40
Meta elementCellular automatonResultantGoodness of fitChemical equationTexture mappingPixelSubject indexingMultiplication signQueue (abstract data type)Graph coloringShape (magazine)Moment (mathematics)Logical constantMathematicsMotion captureInstance (computer science)Medical imagingVideoconferencingObject (grammar)LaptopSocial classFood energyWeb 2.0Pattern languageDimensional analysisDefault (computer science)Functional (mathematics)DistanceInformationScaling (geometry)Phase transitionCASE <Informatik>LogicRevision controlSign (mathematics)Process (computing)Group actionAdditionWebcamFrame problemSpeicherbereinigungMappingGreen's functionRadical (chemistry)Virtual machineWrapper (data mining)Utility softwareConnectivity (graph theory)CodeModule (mathematics)MereologyShader <Informatik>Real numberCubeFile formatComputer animationSource code
54:38
Game theoryCodeLogicLoop (music)Unit testingSystem callExecution unitLecture/Conference
55:50
MereologyDefault (computer science)WebsiteVirtual machineIncidence algebraGroup actionTheoremFormal languageLecture/Conference
56:39
Scripting languageCodeReal numberError messageLecture/Conference
57:36
Task (computing)PlanningTape driveComputer animationLecture/Conference
Transcript: English(auto-generated)
00:11
Hi, everyone. You hear me? Yeah, okay, so how many of you play video games? Great. How many of you develop video games?
00:25
How many of you would like to develop video games? Okay Again, how many of you know this tool? For those of you that don't know this tool. It is the
00:41
Engine, the framework that has been used to develop the following titles. If you are a video game player, you should recognize at least one of them. Gears of War, one of the masterpieces for the current Xbox. Tekken, this is the seventh installment.
01:01
Street Fighter, I hope everyone knows it. The recently announced Darksiders 3. This company started with its own engine with the previous two installments, then moved to a real engine 4. This is not so famous, but it is a kickstarter,
01:22
a successful kickstarter, made by the designer of the original Castlevania series and even the remake of Final Fantasy 7, that is one of the most famous and beautiful games ever made, it is in work with a real engine 4 and
01:41
a lot of more titles. So before starting working with a real engine 4, you need to get it. For those of you that don't know it, a real engine 4 is open source. It's not free software, but it is open source. So you register on the a realengine.com site and then you get access to the GitHub repository where you can clone
02:06
the source code of the whole engine and the editor. You can eventually build it for Linux, Windows, Mac or download binary release. How Epic, the company that makes a real engine, makes money?
02:21
Basically, Epic asks you for the 5% when you start earning more than $3,000, only for games. If you make simulations or films or whether it can be considered a game, it is free for use. Once you download the engine, you get something like that. On the left side,
02:46
you have a list of the actors. An actor is something you can drag on the word. The word is a representation of your game. Each actor can have its logic, so you can program them, and when you hit play, your game starts.
03:02
If those two chair a table, and I don't know what it is, that strange shadow on the table, it does not impress you. I've prepared another screenshot. I think it is way better and can show you how much powerful is Unreal Engine 4. We are programmers, so this is the most interesting question for us. How we program Unreal Engine 4? By default,
03:27
you can program your game logic with C++ that I suppose everyone knows, and blueprints. How many of you know what a blueprint is? Okay, no one, basically.
03:44
Everyone knows C++, right? Okay. This is a blueprint. It is basically visual scripting. On the left, you see an event node. It means when the spacebar event, so when you press the spacebar on your keyboard,
04:07
do something. In this case, it's jump, so your character will start jumping. Even if you're not a programmer, it should be easily understandable, as it causes effect. This is another example of blueprint. We have another event, left control, on your
04:28
keyboard, and then we spawn, so we generate a new actor, in this case a missile. So we basically shot a missile from the position of our player. The get actor transform on the lower left is to extract the
04:43
transform, transforming the gaming word in the linear algebra word is the description of position, rotation, and scaling of an object in the space. This is another example of blueprint, the same one as before, but with a delay node added. With only that node, we can
05:03
implement some form of cooldown for your weapon. So you don't start shooting missiles one after another, but you slow down them by 0.2 seconds. Then you may want to start working with the vector mark. It is very common in game development to start working with vectors, and then you can easily start with this kind of
05:25
spaghetti, or you can do even worse. Now, do not get me wrong. Scripting with blueprint is effectively programming. Blueprints remove only the syntax part of programming. If you respect all of the good
05:45
programming rules, it is very hard to make this kind of mess by following both common sense and good design rule. Blueprints are effectively really cool. This is a debugging session.
06:01
So in real time while your games run, you can see the nodes highlighting, you can stop and you can make black point, you can see the value of variables to change, and so on. Before starting programming a real engine, you have to know a bunch of concepts. UObject is the main class in the sense of C++ that is used by
06:24
the engine. Basically, anything is in the editor and in your game is a subclass of UObject. An actor is a subclass of the object that can be spawned, so generated on your world, so it is effectively being seen on the world.
06:44
A Pawn is a subclass of an actor. A Pawn is a special actor because it can be possessed. By possessed, it means a controller. A controller can be a keyboard, a mouse, a joybed, everything that can abstract the human player, or an AI controller for artificial intelligence.
07:06
A character is a subclass of Pawn that has all of the same characteristics of Pawn, but for anthropomorphic figures. So you have a human with a skeleton that can jump, run, crouch, and so on.
07:22
Then we have component. Component are reusable pieces of code that you can attach to all of the previous classes. If you ever really use some other engine, like Unity or Godot, you may be used to the tick paradigm. So you basically write a single function, update tick,
07:48
it depends on how the editor uses. And in this single function, you make all of your game logic. That function is run 60 times per second, 30 times per second, depending on the frame rate of your game.
08:05
In Unreal Engine 4, using the tick is considered an anti-pattern. You have to rely on events. On the left, you can see a bunch of example events I've defined, so explode, move forward, past, and so on. So you should always follow
08:21
the event paradigm. Why? First of all, for code organization. We have already seen how easy it is to generate spaghetti with blueprints. So, thinking in an evented way should simplify the organization of your code or visual code.
08:42
Events are network friendly. Unreal Engine includes a network stack out of the box, and whenever you use events, that events can be propagated on clients, on the server, broadcasted to all of the clients, to a bunch of clients, and so on. As you have already seen, you get time management for free.
09:03
Events can be triggered by animations too. For example, if you have a running animation, you can trigger an event when the foot of your character is on the ground. I don't know, for spawning particles, or for doing noise. Most important,
09:20
game development is done by a lot of different figures, from programmers, to artists, to sound designers, level designers, architect, and so on. So there are a lot of non-programmers in a team, so following the event pattern should simplify the explanation of your gaming rules to non-programmers.
09:46
When and why should I choose Blueprint over C++? Unfortunately, not all of the C++ low-level API is exposed to Blueprint, only a little subset.
10:01
On the other side, writing shaders, materials in the Unreal Engine Jargo, is extremely easy with the Blueprints, is extremely hard with C++, or with the native languages of shaders. Interfacing with C or C++ library is basically a must.
10:23
It is very rare that all included library in the engines will be enough for your work. Good design, the Blueprints are really easy to read, even for non-programmers. This is a personal
10:42
thing. Blueprints are sometimes too much silent for me during error. Then, this is the central topic. In addition to C++ and Blueprint, we can now program our game logic or eventually script the whole editor with Python.
11:03
It is an open source project. You can freely download it. It's supported on Linux, Mac, and Windows. Why I did it? First of all, my dream was to code a AAA level game with Python instead of C++ or eventually Blueprint.
11:22
Unfortunately, I miserably failed. Why? After the first release of the project, I obviously started checking comments on Reddit, Facebook, and all of the channels where I posted the announcement of the release. I didn't realize that for the game industry,
11:43
coding in C++ is not seen as a problem, as something that slows down your work. Obviously, there are a lot of more or less myopic programmers that really welcome the plugin, but the vast majority of
12:00
macho programmers throw shit on it without thinking twice. Instead of stopping working on it, I started asking my colleagues and the other companies doing games how I should change the direction of the development.
12:21
After a bunch of chats with these people, I came up with this list of tasks. First of all, the main project in the AAA industry is managing tons of assets. There are a lot of artists, musicians that throw assets in the project, and these assets must be managed by the programmer.
12:41
Assets can change, and the programmer has to change again what is mapped to assets, and so on. So scripting, doing orchestration of your development pipeline, it is something really wanted by the game industry, especially before for artists in the 3D areas, modeling, animating, and so on. Python is already present in their development pipeline. A lot of animators scripts their
13:06
working pipeline, so having the ability to use the same scripts, the same code base, even into the game engine, will be really really interesting. Unity testing, as well as functional testing, is not a really diffuse practice in the gaming industry.
13:25
Most of the time, because writing tests in C++ is really hard. If you fail on a test, generally it means a segmentation fault or a crash. You have to restart your work from the beginning. So even having the ability or running unit tests
13:41
was an interesting thing. There is a lot, you know, there is a lot of Python in the scientific and academic world, and data visualization is another interesting topic. So having a really powerful 3D engine will be really useful to show their work.
14:03
Another almost strange thing is simplifying the versioning of their work. A good part of a game is composed by binary data that you cannot version so well.
14:21
So for a lot of areas, it will be very interesting to write code that regenerates the status of your project, and eventually write 3D games with the engine. So I started in August last year.
14:45
I didn't have a high expectation. It was more of a toy project for me. The first step was obviously embedding a Python virtual machine into the engine, having a console for issuing Python commands, and obviously running code and getting output.
15:04
If you don't know, I am the author of the uWSGI project. So I have a really huge experience in embedding Python in C++ application. So it was pretty easy for me to reach this level in basically a couple of hours of development.
15:23
Here you see a console built over the native Unreal Engine graphical user interface. I issued the import sys command, and I get the output. I was really, really, really happy, but then the hard work starts.
15:45
First challenge. I already told you that the most important class in 3D Engine is the UObject one. It is the class from where all of the other ones in here is. So I needed to find a mapping between a UObject native C++ class and a Python native class.
16:08
Following these rules, this mapping must be fast. In this time of the development, I was still trying to use this plugin to write AAA games, so performance was really, really important.
16:24
This mapping must take into account that both Unreal Engine and both Python have their garbage collection. C++ has obviously no garbage collection, but Unreal Engine implemented one. A funny thing in the Unreal Engine structure is that properties of classes and functions of classes
16:46
are built like UObject, so I needed to find a way to map this particular behavior even to Python. Obviously, even on Python, both classes and properties can be objects. Those are the solutions.
17:01
For managing the Python UObject mapping, I've used a C++ 11 map. If you do not know C++, only added in 2011 standard the ability to have dictionaries or hashes.
17:21
When the related Python object goes out of scope, the garbage collector on Python will very probably destroy it. However, I prevent the garbage collector to destroy the UObject mapped to it because Unreal Engine makes a different decision by the Python virtual machine.
17:42
I must ensure that UObject is not destroyed without Unreal Engine knowing it because it will make a crash of the whole editor. Whenever from Python I access the UObject mapping, I always check that the mapping UObject is still valid
18:02
because if it is still not valid, it means Unreal Engine has kicked out that UObject, so the Python mappings maps to something that is no more valid. And then I automatically expose properties and function as Python equivalent.
18:22
After a couple of days, I came up with this. So I can directly call Python script for Unreal Engine narrative event. Next step, there are, I think, hundreds of C++ classes into the editor.
18:46
I think there are thousands, but let's say hundreds of classes into the Unreal Engine editor and engine. Mapping each of them to a Python class would be a huge work,
19:03
a work that I don't want to do. But Unreal Engine exposes a reflection system so I can basically get a reference to a class using a string. My objective was to do something like that. So import the character class like it is a native Python class
19:24
and spawn, on the last line, a character, so an instance of a character into the editor. Don't be scared for the slide. This is how you write extension in C for Python. The interesting part is line 150 that find object camel case function.
19:47
This is how we access the reflection system of Unreal Engine and basically we find object, we find a reference to a class with that name and on line 155, we build the mapping within that object
20:06
and it's Python equivalent. So now I have a way to get a reference to Unreal Engine, a native class using a string in Python. What I do with this, I don't know if you can read it.
20:24
This is basically a hack for faking the import system of Python. Basically, whenever you import, check the last line, from fake importer dot classes, the get at method of figure class is called.
20:47
So I use this technique to fake the import system. So whenever you import from Unreal Engine dot classes, a special get at method is triggered, returning the equivalent classes.
21:02
Incredibly, I get unexpected results, positive unexpected results. The same approach works even for structure. In Unreal Engine, there are objects and there are structures. Structures are passed by value, like in the vast majority of languages and even for enums. So with the same approach, I can import the three main structures of Unreal Engine
21:25
and without knowing it, without having an idea that I already reached that kind of coverage, 80% of the Unreal Engine for reflection system was ready. So with my Python plugin, I was already able to do all of the things
21:43
we can already do with Blueprints. So it was a huge success. Third challenge. Scripting the development pipeline includes even generating Blueprint programmatically.
22:02
I know it could be something strange. It's like generating a language with another language. But this is what people or game developers ask me, especially animators. In Unreal Engine, animators generally add logic to their animation using Blueprints.
22:21
So for them, it's way more easy to use Python to generate those nodes. The results are not really pretty for a Pythonist, but this is how you can build a Blueprints graph with Python.
22:44
What I needed to do to reach that point? I needed to implement Python classes, abstraction classes for graphs. The gray background is there. Nodes.
23:00
Big blocks. And pins. The arrows and the thing that generates spaghetti every time. The current state of the art can be seen in this tutorial. Basically with only Python, you can generate a whole monster with its animation,
23:23
with its brain. I don't remember. You can do really a lot of things in this. It is a pretty huge tutorial. As before, I get unexpected results. This time, not very positive. I get the ability to script animation graphs for free.
23:43
It is good. Beaver Tree are blessed the way in Unreal Engine to write artificial intelligence. They are like a state machine on steroids. Unfortunately, I started getting crashes all over the place.
24:03
What happened? I was obviously working in a way not expected by Epic developers, so I started messing with their code. I started playing at the same level as C++, so every error results in a brutal crash of the engine. I was playing as a virtual editor,
24:25
so basically everything I do from Python should be doable even into the editor. If it is not possible, expect only pain from Unreal Engine. And especially the reflection system does not like messing up with the internal structures.
24:41
After about a year, this system has become really solid. But it requires one year of full-time development. Fourth challenge, unit tests. Here you can see a unit test checking that when I create a new material,
25:01
a new shader for your graphics card, and I give a name to it, that name is effectively put into the engine. It is a pretty easy test case. First of all, when we run tests from the command line with PyTest,
25:23
however you generally do from PyCharmes and so on, a new full Python interpreter is generated on your system. So when the tests end, the whole data generated by them is erased and destroyed. We cannot generate a whole new Python interpreter into Unreal Engine.
25:45
We need a new interpreter for running a single test. So my solution was generating a Python interpreter. The Python CVI allows you to generate a new copy of a Python virtual machine
26:00
that can be destroyed without destroying the main virtual machine. So when you run tests written in Python, a new Python virtual machine is spawned without destroying the original one of the tests. Standard output and standard error are mapped to the Unreal Engine Python console,
26:22
the window we've seen before, and at each run of tests, we do rollback. This is one of the features of the Unreal Engine that shocked me most. It is not documented so well, it is really hidden. I discovered it for a case, I was checking sources and I found it.
26:44
Basically, each operation in the Unreal Engine editor is a transactional one, like in a database system. So I can build my tests and rollback everything they do in the editor. I am free to generate actors into the world,
27:03
check that they are good and then destroy them. Obviously, the gameplay is still something I really want to do in Python. So I've started having these abstractions.
27:23
They are subclasses of actor, pawn, character, Python component and PyAdd. We will see later what is it. They are a wrapper for their native counterpart, but with the ability to call Python code at specific events.
27:46
After a bunch of tests and crashes all over the place, I decided to use a proxy pattern for mapping Python classes to native Unreal Engine object. So basically, you write a native Python class.
28:00
A special attribute is injected in this class, so you get access to the native Python object. This is an example of an actor fully written in Python. The beginPlay method is automatically called when the game starts and the pick method, albeit it is an anti-pattern,
28:23
is called a table refresh of the game. The interesting part here is that self-view object, you can see here right out to the location, the self-view object attribute that is injected by Unreal Engine when you map a native class to an already existing object.
28:45
Why I chose this approach? First of all, for almost safe reloading at each play iteration. Once you load a module or a class into the Python virtual machine, that module is resonant, that code is resonant.
29:01
But during development, you want to change code. When you do web development, generally you restart the application server to see code modification. We can't restart the whole Unreal Engine editor because it requires a bunch of seconds to spawn. So in this way, I can call a simple imp.reload
29:24
from Python to reload the code of a proxy class. In this way, I have a clean separation between Unreal Engine 4 and Python, especially from the garbage collection point of view, as the class is not really tied in, I can destroy one of them without bothering too much the other one.
29:46
This is something I found really interesting, but others don't. Once you have packaged your game, you have your executable with all of your Python scripts into the project directory, you are free to change that Python file
30:03
and basically modify all of the game logic. I understand this is not an interesting thing to do for a game developer, or something that it doesn't want to do from its customer. Obviously, you are free to include not the .py files in your build.
30:22
You can use the compiled one, the bytecode. Another advantage of using a proxy pattern is that I can change during gameplay the class mapped to an object. A character could be mapped at some time of the game to a class and then remapped to another one during gameplay.
30:46
This is another challenge I'm still working on and I'm still failing at it for those people that don't like the proxy pattern. Basically, I want to allow the developer to subclass native Unreal Engine classes
31:05
with Python classes, something a bit strange and immoral for some people. As you can see, as Unreal Engine uses C++ and it's a secret type,
31:21
I've used Python 3 annotation for specifying to Unreal Engine what type of value this function takes. I'm talking about on C phone. Those are the steps I did
31:40
for reaching this broken result. I used the Python annotation for enforcing types. Python became something like a data description language so I used Python to inform Unreal Engine on how to build its internal classes.
32:02
I needed to rely on metaprogramming a lot and a lot of pain for reaching the result. Why Python metaprogramming? Because character, the class from which master inherit,
32:23
is an object, it is not a class. So I need to write some logic to inherit from an object that is not a class and generate a new class. This is how you can fake the inheritance system.
32:40
You can see the funny subclass is a child of fake class that is an instance of fake base class. It could blow your mind, I understand. Unfortunately, as I already said,
33:00
it's still really buggy. It could crash your system. Incredibly, as different behaviors on different engine version, I really have no idea why it happens. It is absolutely a good exercise to know how Unreal Engine internals work.
33:21
I think it is not so worth it because the proxy paradigm is way more safer and solid but it's obviously a lot really, really cool so I think I will try to improve it. What does it need to do to improve the project? I would like to stabilize this sub-classing API.
33:42
I would like to introduce mobile integration because currently you can build games only for Linux, Windows and Mac. I would like a stronger integration with PDB for simplified debugging of Python script into the engine. This is a really huge task
34:01
to cover the whole C++ API. I still have a bunch of doubts. First of all, should they follow in the C part integration of Python, should they follow the Python C coding style, so underscore, lower case,
34:23
or the C++11 with full camel case? In the example code you have seen before, the C implementation of reflection, you have seen two different styles in the code. It is something really, really ugly and currently I've still not made a decision.
34:41
Should they invest more on threads? Threads are supported out of the box. The guild has become not a huge problem. I added a bunch of tricks to reduce its impacts. By default, the binaries you download from the sites have no threads enabled.
35:00
There are builds with threads enabled. When I started the project, a lot of Python developers are really used to Maya or Studio Max want to use Qt in their project, so I invested a lot in easy integration
35:21
between PyQt and PySide into the engine. Nowadays, I have exposed the native graphical user interface of a real engine. It is called Slate. It is a good solution instead of Qt. Slate is based on preprocess or C macros,
35:45
so you can write some kind of readable interface. I've implemented it in Python, trying to give to the developer the same taste of the C++1. This is how you can generate a window in a real engine with a button on its center.
36:04
When the button is clicked, the say hello Python function is called. Into the button, there is a text block with a simple string. Currently, I'm dreaming about more orchestration in the game industry. There is still a lot of manual work.
36:22
I would like to integrate Gherkin for a bigger test. Another dream is to build some kind of domain-specific language for writing shaders, so material and or blueprints, and, obviously, someone using it for Tsubole gameplay.
36:44
This two company, Caden Lightning, is from Los Angeles, and good thing it's from Toronto, Canada. They heavily sponsored the development of the project. There are at least dozens of companies already heavily using it.
37:02
But these two are the ones that put money into the effort. If you are into the scientific academic world and you would like to start drawing your academic research results into a powerful graphical engine, this is a really gentle introduction
37:22
for Pythonista that includes Matplotlib into your game. So, basically, whenever the player walk over one of that colored cube, the graphs on the carpet, it is updated, generating a new pie chart.
37:46
I need to switch laptop because, unfortunately, Unreal Engine is able to melt down my MacBook Pro. So, give me a minute.
39:05
Okay, can you see it? So, this is the third-person template you can generate from the standard Unreal Engine distribution. I have added on the level a bunch of famous phases.
39:24
So, when I play the game, I can move into the world, and see Spark, Gillian Anderson, Wolverine at 800, Sheldon and so on.
39:41
My objective now is to allow the player, the mannequin, to see and recognize faces using facial recognition included in OpenCV, Open Computer Vision, all in Python. First step is adding a Python actor
40:02
into the scene that will implement, hide the sight of our player. So, in the list of actors, I search for pie actor, and I drag it into the scene. On the right,
40:21
you can see the fields where you can specify the model and the class to map to this actor. We will start with this simple code into the init method.
40:45
Better. You initialize a timer of one second. In the gameplay, you simply print some message into the console. And then this value of timer is decreased at every tick,
41:01
and when it reaches zero, it means that time has elapsed and restart counting. It is a very simple code, just to check the Python actor is working. So, the model is eyes first,
41:24
and the class is sight. This is our Python console. I hit play, and on the bottom, you should see on red, relapsing time. So, our Python actor is already working.
41:46
Let's go on. Second step is adding us a special component to our actor. It is the scene capture component 2D. Basically, you can see it as a virtual camera
42:00
that tracks anything it sees and saves it into a texture. A texture is a memory area into your graphics card that can be drawn on the screen. During the initialization, we generate a special texture known as a render target. It is where our capture component
42:22
will write data. I've called it what I'm seeing. I create a texture 512 for 512. You should always try to use the power of true texture when doing game development. It is a transient object. It means it is not an asset
42:41
on your pipeline. This object is destroyed. This special look is called automatically, if defined, before initializing components in your actor. So, we want to add a new component, so we use this look to create it. This is how we add a component
43:01
in our code, the scene capture one. We specify in which texture he has to write data, and we specify that we want the linear values of the colors. This is something I will not want to invest time in explaining to you. It is not that interesting for us.
43:23
So, I go there and change the model to eyes. So, I hit play, and here you can see the actor with a new component, scene capture, on the right. It is already tracking
43:42
what he is seeing. Unfortunately, we cannot see what the player is seeing. Before doing this, we have to fix another problem. This pie actor has no position into the world. Our objective is to attach it over the head of the player.
44:09
Here all is the same. Once the game starts, we get a reference to the pawn used by the player. This is the mannequin,
44:22
the robot working on the screen. And then we attack ourself, the object. This is the injected field into the proxy class. And we attach it to the mesh component of the mannequin into its head bone.
44:41
What is a bone? Here you can see our player, and on the left you see the bones mapped to its mesh. Bones are basically objects mapped to a group of vertices.
45:02
So whenever I move a bone's vertices, follow it. So our objective is to attach our eyes to the head.
45:29
So if we go back, you see the third person character. That is our mannequin. As Pyactor 7 as its child.
45:45
Here you can see the axis of the character, and here you can see the axis of our Python actor. As you can see, it is moving because the head of the mannequin while in idle took some calm breath.
46:03
Okay, next step. We want to develop a head-up display, some kind of 2D graphics onto the screen, so we can see what the player is seeing.
46:23
We have this code, so we spawn a new actor into the scene. A head is another actor. Directly specifying the Python model in the Python class to use. We get a reference to the proxy class mapped to this head, so we can start injecting values
46:42
into it. And we specify into the proxy class which is the texture to draw. And finally, we tell the engine that this new Python head should be the player one, so the default created one is destroyed, so our Python one will be used.
47:02
This is our head. Head proxy class expects you to define a draw head method where you can call graphics primitives like drawing textures, lines, rectangles, and so on, until texture to draw. I remember it is defined in the other class. It is not defined.
47:22
It takes it from the method. If it is defined, it will start drawing it at the zero zero position, so top left with these sides.
47:57
On the top left, you can see what the player is seeing.
48:05
You can see even the balancing when it runs or when it breaths. Basically, we have a texture containing all of the pixels the player is seeing. Next step. For good results,
48:23
OpenCV needs a grayscale image, so we need to generate a grayscale variant of this texture. This is pretty easy. This is the fifth one.
48:43
This time, we import the CV2. It is the wrapper for OpenCV and NumPy because OpenCV uses NumPy for all of the data. In the tick method of our eyes,
49:01
we get the data as a byte array of the texture of the data we are seeing. We generate a new NumPy array giving special dimensions. We have a three-dimensional array with four components, B, G, B, A, and the sides of the texture.
49:23
We use this OpenCV function to convert this texture from B, G, R, A to grayscale. Basically, we have a byte for a single pixel of the image. On the other side, the add class is changed. It is now in the add second module.
49:44
We create another texture with a single channel for color, another transient texture. This time, we need to cooperate with a real engine garbage collector because this texture is never used by a real engine.
50:02
It even does not know it exists. After a bunch of time, it will destroy it. We do not want this, so we inform with this method that we are responsible for destroying it when the Python object is destroyed. In addition to drawing what the player is seeing in full color,
50:21
we draw another texture below it without ignoring transparency channel because we have a single byte for each color. I needed to add these additional values.
50:46
We have the same view, but in grayscale. Now, OpenCV is able to recognize faces from the below image. This is the last version of the code.
51:03
These remain the same. All of the OpenCV logic has been done in the hub. We set up a cascade classifier. It is the way OpenCV uses the algorithm.
51:23
OpenCV uses to detect something. In this case, I have filled it with information about frontal faces. Then, at each iteration, I try to detect faces with this function,
51:45
detect multi-scale, and it will return a quadridimensional array with the information about where the face is.
52:06
You should see green. Whenever it detects faces. Let's see if we can recognize Spock. Here it is.
52:22
It is even recognizing Dana. I push Spock away. People with bears are really hard to recognize, but this time it is taking a few.
52:41
I don't know if it is a good thing, but it is not able to recognize a terminator. I don't know if it is good or bad. It is interesting that it is not able to recognize Saragonor. I think it is because of the goggles,
53:02
the battle goggles that are really strange to recognize for a machine. Last part. By default, Unreal Engine has no features for using the webcam of your mobile phone or your laptop, but OpenCV is able to do it.
53:23
I created this cube, and I added a Python component mapped with the utils webcam class that initializes video capture,
53:41
creates a transient texture with the size of the texture my webcam is able to manage, and I assign that texture to the material, to the shader assigned to the cube. At every frame, I try to capture the image from the webcam.
54:01
I convert it to a suitable format for my graphics card, and I assign the data to my texture. I think you should see me.
54:29
Okay, it is seeing me. Thank you.
54:52
Who has a question for Roberto? I have been working for Crytek for five years,
55:02
and I know what game development is, and I know what pain it is to code game logic in C++, or in LuaScript, and how ugly loop unit tests. Come on, LuaScript is not so painful as C++.
55:20
Yes, yes, but anyway. So I think integrating game logic and coding it in Python is a really great idea, and it is absolutely brilliant. So it is bravo. Thank you. And I wanted to ask you,
55:41
did you have any performance issues, especially in tick methods, calling it an update loop in every frame? Performance issues? Yes, yes. I started having performance issues when I started doing it too much with the GIL. Acquiring and releasing the GIL is one of the most heavy parts
56:04
into the Python virtual machine. So the default build you get from the site have the GIL completely disabled, so you cannot use Python threads, and the impact of Python is not so bad. Last year, Blueprints was an interpreted language,
56:23
and Python was faster than Blueprints. Now Blueprints are compiled to native C++ code, so unfortunately they are faster than Python. Thank you. Another question?
56:48
Just a quick question. Do people interact with Python inside Unreal? Is there only the REPL, or is there also an ability to maybe extend the ribbon bar there on top,
57:01
like putting little scripts into recurring actions? Sorry, I cannot hear you. Oh, sorry, I try again. I was wondering how people would interact with Python inside Unreal. So I saw the REPL, obviously, and is there also a thing planned
57:21
to put code snippets into buttons on the top of the ribbon for users that are maybe not so deep into coding? Just wondering. There you go, isn't it?
57:45
No worries. So I saw the REPL, so people would interact with Unreal in Python via the REPL, and maybe there are some recurring tasks
58:01
people have while working with Unreal. I was wondering if there was also a plan maybe to have some of these buttons that I see on the top to incorporate Python snippets in there? Yeah.
58:28
I imagine. Cool, thanks. Another question now? I don't need to run? Okay, that's fine.