Add to Watchlist

Dynamic Class Generation in Python

216 views

Citation of segment
Embed Code
Purchasing a DVD Cite video

Automated Media Analysis

Beta
Recognized Entities
Speech transcript
OK so we're moving on to the last speaker before lunch so I'm guessing everybody's hungry but please refrain from disclosing the goal is to me is before lunch but please palmapper go talk about dynamical as in recent Python right and also my name is Hanna and I am a software developer at US where I've ever really focused on developing the universe as a caper Python also known as 3 and the area so I will Python-based command to administrative resources and talk to you about dynamic class generation life so when me 1st go over what you should expect this talk person to talk
to you about the basics of dynamic generation mainly what it is and how you go about doing it then i'll talk about the benefits of doing it and when it is appropriate to use it and finally I'm going to build an application from the ground up that uses dynamic frustration in really shows you all the best thing to do so let me start by answering the question what is dynamic class generations what it is is it generates the patterns to generate classes that went time so all the functionality of the specific class so the definition specific classes exist in the source code and the result is it's generated as the program runs and because the cost of an existing source code departure now you need to be generated from some arousal in most cases it's some other data sources such as a schema API so let me show you a simple example of how you can dynamically generate a class here I have a whole world function that takes an object in prints hello world from the edges I also have a class they're called base class that takes and sustained as a parameter insisted as a property now I call type used usually you're familiar using type user takes only 1 argument tells you the type of object but here type can also accept 3 different arguments where the 1st argument is the name of my class the 2nd is a tuple representing inheritance and what this new class that and the 3rd parameter is the dictionary representing the additional properties I want to add to this class and once the call-type it will actually create menu classes such an instantiated as so now I oppose introspect some of these variables here if I look at the the name my class appropriately show my classes next if I try to explain the properties like and sustain go show up all those models and actors that property and also I can call hello world from this instance stated that additional functionality of pretty Hello world from this specific so you probably things like OK that's cool but why should I care about that in a question 1st of all it improve your improve your workload in the sense that you will have you will have to write a amount of Python code to actually add new functionality maybe not meet United have to write I think could at all which is cool 2nd it can improve the reliability record a lot of times how dynamic class generation works out is give a generalized that most logic was that generalized as result it becomes more heavily well used and tested than 30 can actually reduce the physical size of because specific
losses match exists source code and therefore doesn't take up against the size and finally there the
production of a pattern of going to application that uses the later in talk so now entitlements and the downsides of it 1st of all trace back to a little bit difficult the trip to follow mainly because a lot and using a generalized source that so spirit and as a result you have to look through the data the data source that your getting this functionality from to really identify words going wrong secondly for IT support doesn't come right out of box mainly because need the source code to be able to suggest and it simply isn't there so now we go into production level application in terms of but but 3 is that this is the case for Python so allows you to interact with all the different in the CPIs available and but 3 is dynamic and stage of its dynamic in the sense that all of its clients enormous resources are generated at runtime a data-driven in the sense that the functionality for these dynamically generated classes are called from Jason models representing and this it yes and with these 2 qualities allow me as the maintainer of this decay is an efficient work us because but they have rest it's constantly innovating constantly adding new features and services so being able to stay to date and provide all these new features the best case very important but it's very difficult when there's a lot of a guest worker with both 3 in the fact that dynamic in data-driven politics for me is to simply update adjacent model that's packaged in the library and I don't write any pattern code at all and all immediately be able to take up the functionality I can spend my time doing some of the work that that they can be really handle dynamic concentration so now when you do is show you a demo
of what this would look like the very cool so now I'm going to do is open up here and now let's say
areas have a new service called myself so if I try client has fiber my service and I will probably get the runtime you're saying
this is not available so let's actually got Texans so when we you do now
is that and we got a new API model for service and and open up um
this model so you wouldn't cannot looks like and if you look at the
various just this is Amazon E
C 2 using general it's nothing new here and now that I
wanna can use now is that it still alive which share the same data path as noted 3 In
order to add this model on the same data that share support provided for the service model that while they're just opened and then I provide a new CIA was renamed the the service into my
service but that just
copy the file into the correct that but research subset of open up IPython again I can
then use so provided 3 increase try to create new client inappropriately creates and now I can actually just college easy-to-use operations and is going to describe regions called and
appropriately was able to make the call as well so this is really this is really
cool because I did not write a single line that encode enabled to add new functionality so that being said you probably would now wondering when should I
actually considered dynamically generating these classes the 1 big point you probably want to look for is if there exists some canonical source of data where you can actually pull functionality from and better yet if there's more than 1
application possible using this source of data uh class examples where a b and a lot of
times what happens is you have to update the model anyways in order to generate some the server code so being able to use that for the client code you get the bigger the better propre which is also then you can get in the case of databases where you have a defined schema and libraries style exam and are able to use a schema to create a dynamically eyes to interact with this database so now we say
actually build an application from the ground up that uses dynamic class the application on a bill is
a lightweight but it could be it involves having a client interface where it's methods are one-to-one mappings to the various media and this applications we all model-driven there's not going to be any encode that specific to a specific it can happen is to be able valid inference based on models is to be able to 1st responses based on models and it's pronounce something support is not reciprocal so let me go over the
central point as well of were 1st I'm going to ship Shiite write a simple days on the supply and then I'm going to integrate PPI models to pick up input validation response passing then add case specific methods to this application are declining face stuff I make sure it's extensible 2nd inject menu classes and by land actually add more at the so for those of you not
familiar RPC on the briefly go over so do is obviously relies on using comes against a single year where there's not any additional half free-trade if worry about and mostly data is some bodies through the request response silicon request you will specify like a method or parameters you want right and acted in a responsible return result from that method call the now let me go over some sample code of how the system RPC find work out don't worry about looking through deeply right now all walk through with you so now if I was to try to this this is this time with an end point 0 0 so that you can import your all related when I try to make an API chemical before they make ought to specify a method and parameters such that the method gets maps of the method element and the J some payload of sent in the parameters will be mapped to the on only that was and that it was said I can just use requests to just send opposed that you're all with the jails on document and will return to me is the is document as well where you can see here all have the result of multiplying 1 by 2 by 3 together which as you the 6 so it's not that we need to be expanded on this this feels very general there's obviously you don't even need this got this biclustering now you could probably just call request directly you don't have any input validation you have no idea what methods you use what parameters prior the types LCA returns entire response seeking others in the responses of elements talking about with the idea of the request response was in the version adjacency being used
then talk I said to you which is agreement API models since this kind so currently
wherever we can precrime with an for your own generalize call by then this step will be always view is actually take adjacent model loaded and have the new client class consume this model such that I can make a model EPI call which will do is be able to know if its position or keyword arguments except and appropriately print out just the result I 1 and then similarly if I a bypass and inference parameter type sealing the strength to the multiply 1 by you that will runtime error saying this is a touch I expected that so in order to get this working let's talk about the piano here is a simple API scheme that we're going use this is a lot simpler than some other scheme you might see out there like in Web for users or something like swagger of abilistic in this presentation simple so here what do in a schema is you can't define what the end point you're all here is for years EPI and also you can provide operations a new 1 provider in our case when I multiply for now and for each operation you can specify an over over our working uh documentation for that method you can specify what inputs was look like and also with the outputs of specific like as well this but in terms of the amplitude and say what the type is in our case the stimulus and therefore we want to have the additional arguments and you can also specify what the document will tell you when described this as well and you can also do model what what each element in this list will be in this case it's images and you can do the same for the output which you get documentation and also specified well so now we're going to do is actually take this model and then trying to remedy the client of don't worry about trying to read through this right now all over his wall so now what we do is important they get adjacent file into model we now have this model multiplying class consume the model and it's an initializer well do of creates a grammar data in the response which will use the model to both validate in parts responses based off the provided parameters and the API response given that so now what I'll do is continue on and instantiate the inherited kind initializer and then I'll be able to use the make model in the article in which case well 1st do is try to validate the parameters provided using the validated and improve model once we know these are these parameter values will then use the make API call which is inherited from the client to order to actually make a request against the then we will be able to pass the response space of the model and the response given back to us as now about when he's expanded honest still feels to general mainly because they can still completely undocumented we have no idea what methods to invite you have no idea what parameters and without produced in B so let's go fix this by actually
adding address specific methods by dynamically creating them so
only what we have is we note that the model for each station applying the commodity chemicals but by then step will be able to do is now use a factory function to create a new client L actually have these new methods that we want on the client and similarly if we got help on a client we can also get documentation on how what will actually look like so let's go over and code and how this will work so once we will introduce a model and pretty quiet we'll answered initializing similar variables needed for the type of related to call later down the road in which case we just called the class name my client in that and bases we wanted to hear from in this case and also we will create an empty dictionary for word class properties everyone and in what the class so now what we do is actually open up operate the model and look for all the operations available to us and call this helper functions declined with the correct current method will do is actually defining unique at a new function called underscore EPI call in which will be used as used by the extension point so what we to return that defined functions we attach it to be added to the class properties that will provide 2 types which will actually create this new point for us and then with this new point class will stantiated instance of so set now if I call multiply 1 2 3 it's actually crossing out to this newly defined that defined PPI call here where itself is now referring to the client that a attached to the method of a test you and the make model PPI call which is what's called underneath of this function is inherited from the model class but there's 1 big issue the doctrine still not specific so if I try to call help on this firing now goes see exactly what I described which is multiplied is just a proxy activists 104 API call so let's figure out how to do that that's that's actually not too difficult you just add these 2 lines here which were sitting down the name and the or the method that we add to the class properties such that recall help now we will actually pick up the new the new documentation the correct name for the method so by setting the naked under name right here to 1 of the string operation and allows you to override that says the fact that it looks like a proxy people help and by 7 and then adopt you're actually able to set the documentation for that matter that get docstring method for is pretty much what it does is takes operation model and look straight the all the different documentation elements of you remember from before the EPI schema in order to generate the concatenated together a string that has the operations in the type of printer types all returned to 14 so let's talk about when he's extended officer model 1st of all it's not accessible it's an example sense that many of you will support looks something like custom class names for custom inheritance in the sense that we all 1 actually only have to rely on these model the TI
methods we want to be able and new functionality topic so let's talk about step forward in which we make quite sensible let's
start with this sample application where a critic cast flying in a sense that we know that a given as a set of a method and some parameters running at the same result every single time so this actually makes sense that the server every single time do why not just store the map the result memory and return that we needed so with this new point class all pretty new dictionary representing the cash and then I override the make model the API calls such that creating new the consisting of the method arguments and keyword arguments that provide instructors see it he is an operation test and if it is of this print out where traveler retrieving a from and also returned result from the catch than it is now possible will do is actually make a call to the server and the result in store for us so thoroughly worried that we have this logic but there's not really a great way to actually cocaine our test line causes there's no option for us right so but it formula stick this new model it actually CreateClient system can override what the name is of the class 1 use and it's inheritance such that now if I can't find that multiply you can see words receiving the result from and then called again FIL actually say that result in states material from history and then I can also look at what the name is as well and see that they were return my super-smart client as in defined so how to do this is actually pretty straightforward other than updating the signature for this function I just added this these 2 lines which sets of people that know class bases were about In order to do that Our it's not walk through this and I create a new point with the model and the customized the strain could name and the cost basis you can see that my super-smart point gets mapped to the class they type this fall in and class cases it met her request finds tuple gets mapped across spaces when tactical so now I wanna come multiply you can see that it's a verbal using the cash client and when I call that is gonna name ill appropriately use my SupersPort for the string the so as I'm going to be standard here but to be honest based root solanine glaring holes except for the big elephant in the room which is there's only 1 API methods so let's actually go fix them so in this final
step and API such that our
engineers of working server and were hard at work in the end new API and insecure so and in order to actually update this API they have to the models as well in order to generate some server code and as a result we got these new make API models for free and to look at some of these API models so you can see that for ad it's very similar to multiply in a sense it's a list of integers and will return to you is a sum of images of the a single subtract is also the perimeter and where the list of integers and they will return to you in the difference of those images as well as in the so now let's
actually do a demo with us for like so if I have right here but I'm going to do
is I'm going to import a couple helper functions person and for 1 that lets me get my models referred frequently without having to with open up the files and loading it and how important they require function so now if I open if I probably model what I'm going to do 1st is i'm going to open up the old model the 1 that only has the multiplying it so that when I predecline you can see that it only has 3 methods
as that generalize make API call it as a model 1 and also has a multiplier so I call multiply now if I print out uh the correct answer 6 but now patrilocal added something that I will be able to do it because the guy models not up to date here so now let me go fix that and use the new age of the API models I got free so I create this kind again and look at all the
different options that our method that don't the available to me now I can see that adds to do had 1 1 against each other it's going to get me to invite subtract on this instead the program show 0 so this is really cool because I want to have the right anymore Python code
once we once the application was built from the ground up and adding new features and feature is not very difficult at all than alanine gets to the kind of conclusion of this talk where then a class
generation I realize my sample application was super simple it was just adding subtracting multiplying numbers together there's I really even need to be aware that this
but what I want to get at is the fact that we start of an application can filter to the ground up such that in order to add new functionality all required was updating a new model for getting model hopefully you're running on the 1 hand have update this new data sources in order pick up function and that's really powerful especially when you have a positive another applications possibly using this similar model such that 1 updates the model can update and different applications that may be consuming dynamic last iterations produces really
robust coding because it's a generalized other could path and all your logit long after that down that as a result
you did very well tested heavily used code and 1 thing I really don't talk about was features and bug
fixes have a lower much wider spread impact in the sense that right now for the client application I wrote it only supports this RPC protocol if I was said something like rest Jason arrest an awkward protocol
what that opens it up for me is to use to be able to support a bunch different you guys and maybe only running on a protocol and in terms of politics is applied it's a bug in a certain date in a certain code path chances are even on was chartered for 1 functionality if another function is using the same code path politics about in there too so I really hope you guys got
a lot all ideas about how plastics Dana class generation words users where we use it in class we have some ideas and using it in your next that product or even for a product to some electoral thank you all for coming if you want to look at some of like the precision code here is again have repository for that here is about a free repository the 1 I can look at the more nitty-gritty stuff on resources incline generating dynamic generation and all that but of course that's were mostly client bank find generation happens but if we just an approach proxy that to that's if you want actually see how clients created there I recommend checking that out but otherwise I'll be here all week so if you guys have any questions about what 3 but of course the alive or about ATBs come find me all the happy talk about it but that's about it is a if you don't the thank you very much
could you contrast this technique that you showed move with the 2 other ways of doing this and the thing that is using that did not move that have to do organ and to using with the classes yeah so many classes that really something you can actually try to reason with and this is in reality type is just another class right so you can define your own metaclasses if you want some specific functionality is out of it as well so like there's a bunch of different other approaches in order to dynamically generating classes is just type is 1 of 1 that is kind of built that filled with for you right out of box anyone else what questions OK and that's wants to think about
Area
Video game
Topological algebra
Computer animation
Software developer
System administrator
Universe (mathematics)
Right angle
System dynamics
Computer programming
Topological algebra
Code
Run time (program lifecycle phase)
Scientific modelling
Multiplication sign
Source code
Letterpress printing
Frustration
Insertion loss
Parameter (computer programming)
Topological algebra
Workload
Latent heat
n-Tupel
System dynamics
Data type
System dynamics
Inheritance (object-oriented programming)
Physicalism
Instance (computer science)
Cartesian coordinate system
Variable (mathematics)
Functional (mathematics)
Category of being
Computer animation
Personal digital assistant
Logic
Pattern language
Object (grammar)
Data type
Resultant
Row (database)
Service (economics)
Code
Scientific modelling
Multiplication sign
Demo (music)
Source code
Client (computing)
Term (mathematics)
Cuboid
Energy level
System dynamics
Run time (program lifecycle phase)
Product (category theory)
Demo (music)
Concentric
Client (computing)
Bit
Software maintenance
Cartesian coordinate system
Functional (mathematics)
Word
Computer animation
Personal digital assistant
Pattern language
Energy level
Electric generator
Resultant
Computer-assisted translation
Library (computing)
Cognition
Service (economics)
Scaling (geometry)
Scientific modelling
Interactive television
Library catalog
Client (computing)
Blog
Object (grammar)
Information
Data storage device
Fiber (mathematics)
Lambda calculus
Data type
Default (computer science)
Area
Run time (program lifecycle phase)
Service (economics)
Gateway (telecommunications)
Trail
Online help
Interior (topology)
Code
Ext functor
Client (computing)
Point cloud
Event horizon
Computer animation
Synchronization
Physical system
Identity management
Electric generator
Computer-assisted translation
Elasticity (physics)
Service (economics)
Gateway (telecommunications)
Service (economics)
Trail
Scaling (geometry)
Set (mathematics)
Scientific modelling
Interior (topology)
Code
Point cloud
Library catalog
Event horizon
Computer animation
Blog
Order (biology)
Synchronization
Data storage device
Identity management
Computer-assisted translation
Lambda calculus
Elasticity (physics)
Service (economics)
Computer file
Scaling (geometry)
Set (mathematics)
Interactive television
Library catalog
Client (computing)
Ordinary differential equation
Open set
Subset
Blog
Object (grammar)
Operator (mathematics)
Information
Configuration space
Lambda calculus
Data type
Default (computer science)
Service (economics)
System dynamics
Dialect
Trail
Online help
Computer file
Interior (topology)
Scientific modelling
Code
Point cloud
Client (computing)
Ext functor
Event horizon
Computer animation
Synchronization
Identity management
Physical system
Computer-assisted translation
Elasticity (physics)
Point (geometry)
Service (economics)
System dynamics
Online help
Computer file
Interactive television
Source code
Scientific modelling
Ext functor
Client (computing)
Line (geometry)
Canonical ensemble
System call
Dynamical system
Content (media)
Computer animation
Object (grammar)
Information
Configuration space
Physical system
Data type
Default (computer science)
Server (computing)
Code
Multiplication sign
Scientific modelling
Source code
Client (computing)
Cartesian coordinate system
Dynamical system
Computer animation
Database
Personal digital assistant
Database
Order (biology)
Library (computing)
Injektivität
Computer animation
Code
Hypermedia
Building
Scientific modelling
Dependent and independent variables
Interface (computing)
Weight
Client (computing)
Cartesian coordinate system
Inference
Point (geometry)
Addition
Validity (statistics)
Mapping
INTEGRAL
Code
Scientific modelling
Multiplication sign
Element (mathematics)
Sampling (statistics)
Client (computing)
Parameter (computer programming)
Cartesian coordinate system
Revision control
Latent heat
Computer animation
Personal digital assistant
Dependent and independent variables
output
Data type
Resultant
Computer worm
Physical system
Numbering scheme
Presentation of a group
Computer file
Scientific modelling
View (database)
Client (computing)
Function (mathematics)
Parameter (computer programming)
Mereology
Inference
Web 2.0
Medical imaging
Latent heat
Term (mathematics)
Operator (mathematics)
Touch typing
Proxy server
Position operator
Exception handling
Run time (program lifecycle phase)
Spacetime
Validity (statistics)
Element (mathematics)
Scientific modelling
Electronic mailing list
Client (computing)
System call
Computer animation
Personal digital assistant
Internet service provider
Order (biology)
Formal grammar
Dependent and independent variables
output
Right angle
Data type
Resultant
Point (geometry)
Code
Scientific modelling
Online help
Client (computing)
Data dictionary
Workstation
Latent heat
Operator (mathematics)
String (computer science)
Factory (trading post)
Software testing
Office suite
Extension (kinesiology)
Subtraction
Proxy server
Address space
Inheritance (object-oriented programming)
Military base
Element (mathematics)
Line (geometry)
Instance (computer science)
Variable (mathematics)
Functional (mathematics)
System call
Category of being
Latent heat
Word
Computer animation
Personal digital assistant
Factory (trading post)
Order (biology)
Right angle
Figurate number
Data type
Point (geometry)
Read-only memory
Server (computing)
State of matter
Scientific modelling
Multiplication sign
Letterpress printing
Parameter (computer programming)
Client (computing)
Root
Computer configuration
Well-formed formula
String (computer science)
Operator (mathematics)
n-Tupel
Factory (trading post)
Software testing
Physical system
Exception handling
Spacetime
Mapping
Inheritance (object-oriented programming)
Military base
Basis (linear algebra)
Sampling (statistics)
Client (computing)
Set (mathematics)
Line (geometry)
Cartesian coordinate system
Functional (mathematics)
System call
Electronic signature
Word
Computer animation
Data storage device
Logic
Personal digital assistant
Order (biology)
Data type
Resultant
Server (computing)
Code
Scientific modelling
Electronic program guide
Scientific modelling
Electronic mailing list
Medical imaging
Summation
Computer animation
Order (biology)
Integer
Freeware
Subtraction
Resultant
Perimeter
System dynamics
Demo (music)
Computer file
Online help
Scientific modelling
Demo (music)
Interactive television
Heat transfer
Ext functor
Client (computing)
Functional (mathematics)
Content (media)
Computer animation
Object (grammar)
Information
Physical system
Data type
Default (computer science)
Computer programming
Online help
Code
Scientific modelling
Interactive television
Ext functor
Client (computing)
Attribute grammar
Error message
Computer animation
Computer configuration
Object (grammar)
Information
Physical system
Multiplication
Data type
Default (computer science)
Subtraction
Topological algebra
Error message
Computer animation
Object (grammar)
Mathematical singularity
Sampling (statistics)
Attribute grammar
Cartesian coordinate system
Number
Subtraction
Computer animation
Code
Scientific modelling
Order (biology)
Source code
Iteration
Cartesian coordinate system
Functional (mathematics)
Resultant
System dynamics
Computer animation
Code
Bus (computing)
Client (computing)
Communications protocol
Software bug
Word
Product (category theory)
Topological algebra
Computer animation
Code
Term (mathematics)
Repository (publishing)
Client (computing)
Functional (mathematics)
Communications protocol
Proxy server
Software bug
Order (biology)
Cuboid
Self-organization
Right angle
Data type
Functional (mathematics)
Subtraction

Metadata

Formal Metadata

Title Dynamic Class Generation in Python
Title of Series EuroPython 2016
Part Number 4
Number of Parts 169
Author Knapp, Kyle
License CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
DOI 10.5446/21171
Publisher EuroPython
Release Date 2016
Language English

Content Metadata

Subject Area Information technology
Abstract Kyle Knapp - Dynamic Class Generation in Python This talk is about dynamic class generation in python: the practice of writing code that generates classes and their functionality at runtime. It will use boto3, the AWS SDK for Python, as a basis to dive into the basics, the benefits, and the drawbacks to dynamically generating classes. ----- This talk is about the concept of dynamic class generation in python. The whole idea is writing code that generates classes and their functionality at runtime. You now may be asking yourself, “That sounds like a neat trick. Why would I ever generate my classes at runtime?” Here are a few reasons why: - It can decrease the physical size of your code. - It can improve the workflow in adding new functionality. - It can improve reliability of your code. One example where the power of this concept has really been leveraged is in boto3, the AWS SDK for Python. Dynamic class generation has allowed boto3 to become heavily data driven such that most of its classes and methods are generated based off JSON models representing aspects of an AWS service’s API. For example, to add support for a new AWS service API in boto3, just plop in a JSON file into the library with no additional Python code required. Using lessons and techniques drawn from developing boto3, this talk will dive into the following topics related to dynamic class generation: - The basics of dynamic class generation such as how to effectively dynamically generate classes. - How to overcome some of the challenges of dynamic class generation. - The tradeoffs in dynamically generating classes and discussion on when it is appropriate.

Recommendations

Loading...
Feedback
AV-Portal 3.5.0 (cb7a58240982536f976b3fae0db2d7d34ae7e46b)

Timings

  675 ms - page object