Abstract Base Classes: a smart use of metaclasses
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 160 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/33740 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 201784 / 160
10
14
17
19
21
32
37
39
40
41
43
46
54
57
70
73
85
89
92
95
98
99
102
103
108
113
114
115
119
121
122
130
135
136
141
142
143
146
149
153
157
158
00:00
Inheritance (object-oriented programming)AbstractionEnergy levelMoment (mathematics)Natural languageSocial classLecture/Conference
00:44
Polymorphism (materials science)Quadratic equationInterface (computing)Social classAbelian categoryAbstractionEnergy levelBitFormal languagePolymorphism (materials science)Energy levelComputer programmingInterface (computing)Social classUltraviolet photoelectron spectroscopySheaf (mathematics)Parameter (computer programming)Object-oriented programmingAbelian categoryModule (mathematics)Programmer (hardware)Meta element
01:38
Data structureObject-oriented programmingData structureSocial classParameter (computer programming)JSONComputer animationLecture/Conference
02:11
Data structureWorld Wide Web ConsortiumSocial classInstance (computer science)DampingElement (mathematics)QuicksortJSONComputer animation
02:38
Data structureSoftware testingWordRight angleSlide ruleArmProgramming paradigmJSONXMLUML
03:05
Data structureSoftware testingSocial classView (database)Inheritance (object-oriented programming)SpacetimeQuicksortSocial classFunctional (mathematics)State of matterType theoryComputer animation
03:45
Electronic mailing listSocial classSocial classView (database)Functional (mathematics)
04:19
Object (grammar)CodeComplex (psychology)Line (geometry)Attribute grammarError messageCodeJSONUML
05:08
Data structurePrice indexObject (grammar)Data structureSlide ruleJSONXMLUML
05:33
Message passingObject (grammar)Social classType theoryInstance (computer science)Electronic mailing listTupleCodeSummierbarkeitCASE <Informatik>Phase transitionParameter (computer programming)
06:28
MereologyInclusion mapString (computer science)TuplePerfect groupRevision controlSequenceObject (grammar)String (computer science)CASE <Informatik>Arithmetic meanSocial classOperator (mathematics)Optical disc driveRevision controlFunctional (mathematics)CodeData conversionInstance (computer science)Core dumpSoftware developerMechanism designElectronic mailing listGoodness of fitJSONXMLUML
08:32
Revision controlSocial classRevision controlBasis <Mathematik>MereologyObject (grammar)Social classBitAttribute grammarInstance (computer science)Inheritance (object-oriented programming)Line (geometry)JSONXMLUML
09:12
Revision controlImplementationSocial classLemma (mathematics)Inheritance (object-oriented programming)Real numberObject (grammar)Social classInstance (computer science)Link (knot theory)Module (mathematics)String (computer science)Key (cryptography)CodeInheritance (object-oriented programming)Loop (music)Library (computing)Electronic mailing listArithmetic meanSummierbarkeitEnergy levelAttribute grammar2 (number)Multiplication signBasis <Mathematik>Greatest elementObject-oriented programmingProjective planeRun time (program lifecycle phase)Right angleType theoryConstructor (object-oriented programming)InternetworkingSubgraphSequencePower (physics)Condition numberAsynchronous Transfer ModeMereologyMappingVideo gameJSONXMLUML
13:04
Inheritance (object-oriented programming)Social classTupleQuadratic equationAbstractionAbelian categoryMaxima and minimaEnergy levelBuildingoutputMenu (computing)CASE <Informatik>Meta elementInstance (computer science)Lemma (mathematics)Interface (computing)Inheritance (object-oriented programming)Social classInstance (computer science)Directed graphSequenceCorrelation and dependenceGreatest elementCompilerCross-correlationForcing (mathematics)Object (grammar)Keyboard shortcutQuicksortAbstractionString (computer science)Right angleRange (statistics)Interface (computing)Module (mathematics)Game theoryMultiplication signAbelian categorySystem callInterface (computing)View (database)Java appletSlide ruleDifferent (Kate Ryan album)Workstation <Musikinstrument>Data structureMereologyCASE <Informatik>Monoidal categoryTupleType theoryDistanceAttribute grammarAssociative propertyParameter (computer programming)Link (knot theory)Object-oriented programmingProgrammer (hardware)Execution unitCoefficient of determinationMappingConcentricMechanism designPower (physics)PlanningMeta elementTemplate (C++)CodePattern languageElectronic mailing listJSON
22:49
AbstractionMeta elementInterface (computing)Quadratic equationUser interfaceSocial classTypprüfungInterface (computing)ImplementationModule (mathematics)CuboidJSONXMLUML
23:21
Quadratic equationInterface (computing)Meta elementSocial classSocial classCASE <Informatik>Electronic mailing listAbelian categorySequenceFlagString (computer science)ImplementationInterface (computing)Inheritance (object-oriented programming)Category of beingDefault (computer science)Formal languageCorrelation and dependenceType theoryBitAbsolute valueAbstractionTypprüfung
26:23
Inheritance (object-oriented programming)Energy levelInformationSocial classLecture/Conference
27:07
TupleoutputAbstractionSocial classBuildingFigurate numberRight angleSequenceSocial classObject (grammar)TriangleDigital electronicsInformationEuler anglesFunctional (mathematics)Attribute grammar
28:00
Meta elementSocial classRight angleCASE <Informatik>MultiplicationInheritance (object-oriented programming)Revision controlLecture/Conference
28:39
outputBuildingAbstractionSocial classPoint (geometry)BitInheritance (object-oriented programming)Right angleNetwork topology
29:22
TheoryMultiplicationInheritance (object-oriented programming)Lecture/Conference
Transcript: English(auto-generated)
00:09
Thank you, and thank you for being here. Yeah, so today I will talk about Asper, asper-based classes and why I think they are useful and a useful thing to know in Python.
00:24
First of all, that's me, too much text, so you have a recap at the end. I want just to point out that I also work sometimes and I am a DevOps. At the moment, I'm just doing a lot of
00:42
AWS and GeneX more than Python. Today's talk is divided into three levels or sections. So during level one, I will talk about polymorphism, behaviors and delegation and collections. So some concepts that I think
01:03
we should know as Python programmers, as programmers in general, and collections is a module that is worth knowing in Python. In level two, I will talk about registering Asper-based classes, finally, and categories. In level three,
01:21
I will go a bit deeper into the topic and talk about how you build your own Asper-based classes, meta classes, and how to use them as interfaces. So Python is a language that pushes the object-oriented programming
01:45
methodology a lot. So one of the tenets of object-oriented programming is the behavior of a structure. So you, we as Python programmers, we should consider behavior of classes more than the structure. What do I mean?
02:04
That at the end of the day, I'm not interested in knowing if something is a list or other things, but I'm interested in knowing if it behaves like a list. So I'm interested in knowing if your class or your instance
02:20
can be sliced, if I can append elements. I don't, I don't, I'm not interested, sorry, in knowing if you inherit from list, or if you implement it from scratch. And this is a reason why in Python we push the EAFP
02:43
paradigm a lot, so it's easier to ask for forgiveness than permission, right? So we try and accept, we try to use something and then we accept, because we are not checking if some OBJ in this slide is subscriptable, you know, if it's a list, if it's something that can be sliced, no.
03:05
We just try it, and if it doesn't work, we do something. And this is, and we do this because we're interested in the behavior. This is an example of behavior. I have two classes, ListView and ItemView, sort of like Django views.
03:25
They inherit from a parent class, which is View, and the parent class doesn't define, sorry, I didn't have enough space in the slide, but the parent class doesn't define the status function, the status method.
03:41
And I implement it directly in the ListView and the ItemView. So there is no way for me to know, to say that ListView and ItemView implement the function. I mean, I cannot say they inherit from View.
04:00
This is not the problem, because View doesn't define the status class, the status method. So this is an example of behavior for me. Those two classes behave in some way. They provide a method. So the question is, how do we check how an object behaves?
04:21
And one thing that we can do is this one. So we can try and call the method, for example. Please note that I'm not calling status. I don't have the parentheses. I'm just asking for the method. Okay, so I'm doing it because I hope, well, I hope, I know that Python
04:44
raises the attribute error if the sumobj doesn't contain the status method, right? But I'm not actually using it. So this is, well, this works. I don't really like it, because it's like three lines of code
05:00
to just perform one check and doesn't, it actually doesn't do anything. More than that, if I have to check a complex behavior, like for example, my object behaves like a list, because I have a complex algorithm, which I have to slice, I have to append things and whatever. I have to check three methods and possibly more than that.
05:22
And this is, well, ugly. Unmaintainable. And I'm wrong, because I am checking the structure and well, in Python there are ways to provide methods without actually putting them in the classes. So I would say this is in general not a good solution. I don't like it as a programmer.
05:48
The temptation we have, and probably already used it a lot, is to use easy instance. So I can say if some obj is an instance of list. In this case, I actually check that my obj behaves like a list. It is a list.
06:06
This is wrong, because it is checking the type of the object. What if I implement a new class that behaves like a list, but it's not a list? This fails. This doesn't pass.
06:22
But the code works. More than that, if I start saying, well, I want an object which is a list or a tuple or a string, because, you know, I am not just checking for a mutable sequence in this case. I mean checking for something that can be sliced. And the three things, list, string and tuple, can be sliced. And then he comes and says, oh, I invented a new tuple, or whatever,
06:47
and I have to add it. So it becomes maintainable very quickly. The perfect solution, in my opinion, would be to have something like behaves like. It's a function that says, yes, your obj behaves like list behavior.
07:05
Well, list behavior is a class, in my dream, that describes a behavior. Okay, that says, yeah, performs this operation, does this and that. Okay, good news, we have something like that.
07:23
First of all, Python is based on delegation. This is a very important thing to know about Python, that a lot of things in Python are implemented by delegating things. For example, you are familiar, I think, with magic methods of Python, like dunder, add, dunder, contains, whatever.
07:45
This is a delegation, because I'm not providing some code in Python core that says that object can be sliced, or contains something, or can be added. It's the object itself, the class, in this case, that provides some method that says, yeah, it can be added to
08:03
whatever, or it can be sliced. And this is important, because actually it's the mechanism that core developers leveraged to implement a solution to my problem, so to my behavior checking function.
08:21
This is the original version of is instance, some Python versions ago. Well, a very simplified version, obviously, but the concept is that what I'm doing is to check. I'm passing a class and then checking that the class is in the basis of my object. Basis is an internal attribute that lists all the parents, okay?
08:46
Actually, the original version is a bit more complex, because we have to somehow climb to the inheritance tree, okay? But this is the concept. I'm checking, I'm saying, yeah, Python does this. Is instance checks for the presence of the class.
09:04
It was changed to a new version, which is based on delegation. Again, a simplified version. Is instance now says that object is an instance of a class. If that object says yes, I am an instance of that class. So the object, sorry, the object, the class says,
09:25
yeah, a method which is called instance check that accepts your object and checks for something. And that checks for something is extendable. I can create new algorithms, okay? So this is not provided by Python in a core
09:41
code. So now this is possible. It is possible to say if is instance sum of vj this behavior, because is instance uses the internal instance check method of list behavior.
10:00
We just need to create some class like this behavior that can check for the behavior, right? Enter collections, and collections is a module that you can find in the standard library. Yeah, it's a module of classes that, as I wrote, that represent interesting behaviors like mutable sequence, sequence,
10:26
mapping, and whatever. Okay, go and check the documentation. This is an example. I import collection. I say is instance a string of a sequence, and the string is a sequence, can be sliced, for example.
10:40
So the check is true, but a string is not a mapping, is not a dictionary, right? There's no key value pairs inside. So the check is false. And the link there is the documentation of the collections module. My advice is to go check it and start using it, because it's a very powerful module.
11:00
It doesn't happen that often in Python, that you have to check, really check, different, very different behaviors. But sometimes, I had my code, the need of checking, for example, if something was a string or a list. It happens sometimes with APIs. I am accepting a
11:23
simple object, a string, a text, or a list of texts. And you cannot tell them apart by just trying to slice them, because if you create a for loop on your parameter, the for loop on the string works, because the string is sliceable.
11:44
Okay, so to level two. Yeah. What is the meaning of being a subclass in Python? So if you know some Python, you could say, well, being a subclass is something that you can do when you inherit from another class.
12:05
And actually, this is true. It's one of the two methods. This is what we call real subclass. You inherit from a class. So child class inherits from parent class. And the relationship between the two classes is very important.
12:23
It's on the bottom. So child class knows parent class. If you have an instance of child class, you can go and check around time that parent class is the parent. It's in the basis attribute. But parent class doesn't know anything about child class.
12:42
It's a responsible parent. Okay, just spawns children and doesn't know anything about them. So you cannot go to parent class and ask, can you tell me all your children in my code base? No, it doesn't work. The second method is to be a virtual subclass. This could be new to you.
13:03
And actually, it's called registering in the object-oriented world in Python as well. What you do is to register, or register probably, the child class into the parent class. And in this case, the relationship is the opposite. Parent class knows that child class is a child class.
13:24
Sorry, pun intended. But child class doesn't know anything about parent. You are just assigning child class to a parent. And this is important because classes that can register other classes are called abstract-based classes.
13:41
So this is the definition for today's talk. An abstract-based class is something that can register another class. And in the collections module code base, you can go and check it on GitHub. I found this code, these lines, which are very interesting because it says sequence, which is a class, an abstract-based class defined in that module,
14:05
says I register tuple or tuple, I register string and I register range, for example. So those three things are sequences because sequence registers them. And a list is also registered as a mutable sequence, which is not, a string is not a mutable sequence in Python.
14:30
One very important thing to remember is that registering is a promise. So Python does no absolutely no check.
14:42
So if you register, sorry, I go back, if you say sequence register tuple, so there is no check done by Python about tuple. So tuple provides some methods, tuple has some attributes. No, it's really just a promise.
15:04
An example of this. Here I import collections. I define a class, my class, which is empty and doesn't inherit from anything but object. So it's pretty empty. If I issue a subclass, my class of collection sequence, obviously I get a false.
15:23
Luckily, this is not a list. But here the trick. I register it. So I say collection sequence register my class. I say my class behaves like a sequence and Python dutifully says it's true.
15:43
Okay, so it's a very powerful mechanism, but with great power comes great responsibility, right? Absol based classes are categories. Just labels. Okay, tags if you want. It's a way to say yes, this class behaves like that, but it's just, you know,
16:02
it's a promise. I'm tagging my class as a sequence or a mutable dictionary or mapping or whatever. You are promising yourself or the other programmers that your class is behaving like you want, like you promised.
16:21
On to level three. How do you build your own Absol based classes? And this is straight from the docs because the example is so good that it's not worth implementing it. So from ABC, which is the module that defines Absol based classes, I import
16:41
abc meta, which is a meta class. More on that in a while. I define a class, which is just a plain empty class, my abc, and I define the meta class of that class is abc meta. It's just code, okay? My class can be empty, and then I can use magically, sort of, use
17:05
register. So I can say, for example, my abc register tuple. And what happens is that I can say that tuple is a subclass of my abc because I registered it. And I can say that an instance of tuple, like the two parentheses there, are an instance of my abc.
17:25
So it's pretty easy to define an abc. Why meta classes? First of all, don't panic. Okay, there's a reason why I wear this t-shirt today.
17:41
Meta classes are used by Python programmers to scare children in the night if they don't go to bed, right? So I want to show you that meta classes are not complex. Not more difficult than classes. Here I have a very simple
18:01
Python code, I think. It's a class definition, an instantiation of that class, and a use of the instance. So book is a class that defines just an attribute answer, 42. When I instantiate the book, I can use the instance and say, please instance,
18:21
give me the answer, and the answer is 42. Pretty easy. I think everyone here understands this code. This is the user of a meta class. I define the meta class, which is a class. It narrates from type, has an init with some parameters, and I can define something inside.
18:44
So for example, set for answer 42. Sounds familiar? I can link a class and a meta class. So I can say my class book has a meta class. And I can use the class and say now, please class, give me the answer, and the answer is again 42.
19:05
I want to compare the two. So on the left, you have the class instantiation. On the right, you have the meta class definition. And I think they are pretty much the same thing. So you have a definition of a class or a meta class.
19:23
You have a use of the class or the meta class, and you have an instance of the thing. Why? Because meta classes, so classes in pattern are ideas, right? So it's something that is sort of a template of a
19:42
concrete instance. And a meta class is the same thing, is the template of a class. I want a class to behave in some way, the class, not the instance. I create a meta class. I think it's pretty easy. If you know how to create an instance from a class,
20:00
you know how to create a class from a meta class. And why? Yeah, it is simple in my opinion. Why are we using meta classes? So, well, for instance, because we need the classes to contain this register method.
20:20
We started from the need of having a register method in the class. We want the class to be able to register other classes because we want to tag them, right, like a stamp. And meta classes can put things into the class. So we use the meta class to say, now I get my ABC,
20:41
my ABC class, and they put inside the register method. And magically, I have it because of the meta class. Once Alex Martelli, who is here, asked me, why don't we use decorators, for example, can be done. I decorate my ABC class
21:05
and I put the register method inside. Well, decorators can be inherited, can't be inherited. So if you inherit from a class that is decorated, you don't inherit the decorator, but you inherit the meta class.
21:21
So meta classes are more powerful in this case than decorators. Oh, yeah, sorry, I have a slide on that. You inherit the meta class. Here you see I define my ABC, which is an ABC meta, and then other ABC, which inherits from my ABC.
21:43
And I can use other ABC just like I used my ABC. So inheritance works here. It's important. Last two slides are about different use of abstract-based classes. I'm not very fond of it, but it's worth mentioning it because it's in the documentation.
22:04
You can use abstract-based classes to build interfaces. If you are familiar with other object-oriented programming languages like Java, for example, you know that there's a concept called interface, which is something that enforces a structure of the object. So for example, you say my class
22:23
has to contain the status method. In Java, if you do this, the compiler complains if you don't include the method, so it doesn't compile. In Python, well, we for many years we didn't have that, and now we can do it with the abstract method decoration,
22:46
decorator, sorry, which is defined in the ABC module. So you just define your class like you did before. It's an ABC meta, okay? And you can put an abstract method, which is
23:01
something that gets checked when you instantiate the class. So in the red box, you see what happens if you try to instantiate the status interface. It says type error because you cannot instantiate an abstract class. Why is not abstract class? Because it doesn't contain an implementation of the status method. It's flagged as an abstract method.
23:27
If you inherit from status interface, in this case, and you don't define the status method, Python complains with a type error saying you cannot instantiate it. You have to do it.
23:42
I'm not particularly fond of it, but it's my personal opinion because I think Python is a very, very dynamic language. So I strongly believe in providing behaviors, flagging things, and not enforcing the presence of methods, but it can be useful.
24:02
So if you want to use it, go and check the documentation because there are other decorators, abstract property, for example, attribute, and so on. And that's all.
24:23
Questions? Was it too fast? Sorry. Hi, thanks for the talk.
24:42
In the beginning, you briefly mentioned that it's not possible to Can you hear me? Okay, cool. In the beginning, you mentioned that it's not as easy to distinguish between string-like things and list-like things. You mentioned an API example that takes either a string or a list of strings, and you said it's not possible, and then you went a bit deeper. Can you
25:05
give advice on what is a proper approach to do this without falling back to like the ugly type checks, as you said? Yeah, well actually it's an example of well I had to implement it and you can do it
25:23
Yeah, for example, if you use collection sequence, that doesn't work, right, because the string and the list are both sequences. You can use mutable sequence because list is mutable and string is not. Obviously if you
25:42
need to tell apart two things that are mutable and sequences, you cannot use collections, right? So if you define something which is like a list but is different and you want to tell a list and your class apart, you have to define your own absolute base class and say, okay, I flagged my, I labeled my class as I don't know what
26:07
my category and then I check that. From the collections you can do it with a mutable sequence. So yeah, multiple mutable sequence. It's pretty easy to do.
26:22
Also, thanks for your talk. I wanted to ask when you create a child class, then the parent class gets put in the MRO. And I was wondering when you register, where is this information
26:43
internally held in the class? If you know. I don't know. Actually, was my question not clear or you didn't? So if I understood the question, your question is, when I register here,
27:04
where is the concept, the information about registering? Well, I don't know exactly where, but it's inside my ABC class. So in Python, classes are objects. So you can do, for example,
27:22
my ABC dot done the name and you get the name of the class. And inside there, actually, I don't have an answer. So I don't know the name of the attribute. When I was investigating Python object, I found there as a very useful function. So try and go and check it.
27:41
I tried, I didn't find it. That's why I'm asking. Okay. I put there on the sequence and I was trying to find like when you register, but I didn't, well, we can do it later and try to figure out. Very interesting. Thank you.
28:02
Anyone else? If not, I've got one question. Okay, so back to the question of why why a meta class and not a decorator. Have you thought about why just not make register a class method and then inherit instead of having a meta class maybe?
28:24
Well, I think it's because you want to, okay, in Python, you can have multiple inheritance, right? It's a hell. So if it's fine for very simple cases, but when you try to have too many parents,
28:44
the problem is that you don't, you know theoretically the methods of a solution order, right? But in practice, it's sometimes it's very complex. If you use Django, sometimes with mixed scenes, you really don't know who defined that method. So the problem with
29:04
inheritance is that you create very big inheritance trees. So I think the point is to avoid pushing further inheritance, which is already a bit too abused. Sometimes we forget composition, you know, so
29:22
it could be possible. Yeah in theory, well in practice as well, but practically leads to the use of multiple inheritance and I like they don't take took this this way. Okay, thank you. Any other questions?
29:42
If not, let's have a big hand for our speaker. Thank you.