Enhancing Decorators with Type Annotations: Techniques and Best Practices
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 131 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/69432 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
00:00
Complex (psychology)SoftwareOperations support systemSocial classEndliche ModelltheorieCodeAddress spaceSoftware repositoryUniform resource locatorSample (statistics)Sheaf (mathematics)Data structureClient (computing)System identificationElectric currentCommunications protocolParameter (computer programming)Function (mathematics)Pauli exclusion principleProcess (computing)2 (number)CAN busError messageInterface (computing)ImplementationFinite state transducerDependent and independent variablesDifferent (Kate Ryan album)Wrapper (data mining)Object (grammar)Variable (mathematics)Electronic signatureMotion captureString (computer science)Service-oriented architectureSynchronizationLevel (video gaming)Local GroupIBM Systems Application ArchitectureCodierung <Programmierung>ConsistencyAliasingGeneric programmingTupleArithmetic meanWeb pagePlug-in (computing)System callQuery languageKey (cryptography)Parameter (computer programming)Server (computing)Library (computing)String (computer science)Object (grammar)Table (information)MereologyClient (computing)Decision theoryMessage passingOrder (biology)DiagramLogicFunctional (mathematics)Remote procedure callConstructor (object-oriented programming)Dot productSlide ruleComputer programmingProbability density functionCodeCASE <Informatik>Functional (mathematics)Multiplication signGroup actionDependent and independent variablesLevel (video gaming)Uniform resource locatorPauli exclusion principleInstance (computer science)ResultantLetterpress printingSocial classElectronic signatureCommunications protocolVariable (mathematics)AreaMathematicsSoftware repositoryAdditionPattern languageElectronic mailing list2 (number)Lambda calculusException handlingSoftware engineeringQR codeData modelTrigonometric functionsWeb browserEqualiser (mathematics)Software-defined radioDoubling the cubeDegree (graph theory)Spherical capStandard deviationTupleStatement (computer science)InformationModule (mathematics)Content (media)Wrapper (data mining)Goodness of fitProcess (computing)Data structureSheaf (mathematics)Computer fileWebsiteAugmented realityCore dumpInterface (computing)Mobile app9K33 OsaEllipseOpen setPosition operatorTraffic reportingError messageNatural numberProduct (business)Router (computing)Form (programming)Attribute grammarPatch (Unix)Different (Kate Ryan album)LogarithmSoftware testingLoginState of matterGeneric programmingSoftware bugQuicksortPoint (geometry)Computer animation
Transcript: English(auto-generated)
00:04
So today's goal, to understand the complex decorator with the type annotation, and to prevent and detect the bugs earlier, and best practice for the typing decorator. About me, I work out the talk table as a software
00:24
engineer in Tokyo, Japan. And also, I contribute the OSS stuff. For example, the PyCharm plugin for the PyDramatic and the RAF. And I'm creating the code generator for the PyDramatic and the other data model
00:42
from the JSON schema, OpenAPI. Please check my GitHub page. OK, the first, how I came up there with this talk. So motivation, the type hinting is very valuable.
01:04
But it's challenging with the decorator. Key question, how to define the colorable type without the ellipsis, dot, dot, dot. And how to manage their function with flexible argument.
01:21
Sometime the decorator change their argument in. And the way to find the best practice. Yeah, I feel it's difficult to check the best practices for the decorator and the callback there.
01:40
And today, I will address this issue and enhance the decorator with the type annotation. So before jumping the contents, I will explain about the structure of that section. So first, I introduce the feature, like a PIP.
02:04
And they show the example code. And they identify the program in the current code. And they apply the feature to fix the code and the recap. And I have already uploaded the slide as a PDF.
02:23
And they also push the Python code in the GitHub repo. They pre-check this QR code page. I will show you again the same URL, the QR code, the end of the decision.
02:41
And I explain this concept by creating the logo decorator for the HTTP client. And this is agenda, the features. Let's start the basic decorator.
03:02
So decorator is a very cool feature, cool syntax in the Python for higher order functions. Processing that can be added before or other function code. This is the PIP for the decorator. This diagram explains how to work the decorator.
03:25
This is the decorator, the internet things. So first, we can put some logic before call the wrap function. For example, print the before running func. After that, call the wrap function.
03:42
And after that, we can call another logic again. Let's create the logo decorator for HTTP client. I feel that this use case is very useful. There's some time that we want
04:01
to log some of the HTTP code and additional information. The requirements, log the before and after the execution of the function. And the decorator should be designed
04:21
for the HTTP client function. And use a custom remote logo class for the logging. So this is the, yeah, I implement this logic. Yeah, please look the roga.sendlog.
04:43
Send the log message to the external server. Imagine some of their files that provide some SDK. And this SDK provides the send log method. It's too easy to send the log message
05:01
to the external server. And call the function. Yeah, maybe call the get or patch or post. After that, send log again. Okay, this is very simple, the HTTP call function.
05:21
Import the request and define the call URL, the argument as the URL, as strings. And call the request get. And return the response.
05:40
The call URL, the HTTP, yeah. And here is the example, remote logo class. This is the dummy class, but it's very simple. Constructor gets their name and the log group and the log level. And has the send log method.
06:04
This method expects the structure, the logs, by the args and the keyword args. So next, how to use this class. The instantiate, the roga name test,
06:23
and the log group admin, and the log level zero. And the call the send log, the message call hello and the user ID one. And I expect to send such a JSON object to the server.
06:46
Yeah, let's apply this remote roga in the degrader. The first instance ate the remote roga as the roga.
07:01
And the call the send log and the run function. And after that, the send log, this time, send the result to the server. And this degrader name is the other logging
07:23
and the upright previous example, call URL. And this degrader with the function, the run like that is the send the log and the call HTTP.
07:42
After that, send the result to the external server. I introduced a cool feature. It's not the newest one, but it's very cool. So we can enforce the keyword argument after the asterisk.
08:02
I pick up the example from PEP. So this function has three arguments, A, B, and key. This is a good case. The first is one. Second is two. And the third, there's a key around the keyword argument.
08:25
But this pattern raises the exception because this example has the asterisk after the B. But this last code lambda doesn't
08:42
have any keyword argument. We should use the lambda as the keyword argument. OK, let's apply this logic, this example. What is the problem with this degrader?
09:01
So this degrader has two arguments, signature, the group and the log level. But if we use the degrader, we can't remember all the arguments. So imagine in the production, the degrader
09:23
may have a lot of the arguments, 10 or 15 or 12. We can't remember all the things. And if we put this asterisk after the group,
09:41
but in this case, the code without the keyword argument was zero. If we run this code, raise error, it's a very simple program. And we should use the label as a keyword.
10:05
It succeeds. Yeah, recap. We can enforce keyword argument with asterisk. This feature used in the degrader and the other traditional function. It's very variable.
10:22
We should use it. Next, the typing protocol. Python has the duck typing. But this typing defines the interface as class.
10:43
Please look at this example. For example, a response like the class has the status, the attribute, and the JSON method. The inherited protocol. So it means this object has the JSON method and the status
11:05
code. It's very simple. Also, we can define the function by using the protocol. We can define the call with the argument URL timeout
11:23
with the type and the return type. It's the same. So what's different? The callable versus protocol. So pre-check the callable. Callable, we need the type areas to define the type.
11:43
And the callable and the str, the fraud. But we can't define keyword name. But the protocol, we can use to create a name, like a URL timeout.
12:01
It's very variable. Also, we can use some library for the common module. So in this case, someone want to use the request as the HTTP client.
12:20
But other people want to use their HTTPX. It's a very modern one. So pre-check the import statement. Both library has the response. The response object has the same attribute, status code. But it's not the same type.
12:42
In this case, we can use the protocol, the response-like object. This class has the status code. And we can apply the response-like as a return type for the request and the HTTPX.
13:02
It's good for the typing. And in the degrader, we can put the response-like object as a return value. We kept the work with the multiple library
13:25
for the request and HTTPX, the type safe, and ensure the response object have a status code. Next, param spec. We can, the flexible signature handling,
13:43
capture the signature of the function to write the degrader that are more flexible, ensure that the function used with the degrader maintain the original type annotation. Pre-check this example I picked up from the PIP.
14:07
I forecast the important part. Import the param spec and define P by using the param spec with P. And we can use this P in this degrader.
14:24
P.args, it means the positional argument. P.keyword argument, it means the keyword argument. Yeah, pre-check. Usually, we use the callable.
14:40
But we can't understand the ellipsis, no meaning. And there, in the wrapper side, type is any and any. But we can use the param spec in this place. It's very cool.
15:00
We can define the query. We kept that we can define the query type with the param spec for the function argument. Next, concatenate. This is very useful when we use to adjust
15:25
some of the function signature, so adding or removing the specific argument while keeping the reset of the signature intact. Please look at the example from the PIP.
15:40
So this function there is very useful. Get the callable, that's the first argument. Please look at the callable first argument. Concatenate the request and the P. And in the degrader, create the new request object
16:04
and add this as the first argument. And in the definition side, we define the first argument request as the request.
16:20
But yeah, this is the callable side. So the first argument is 1, not request, and add the A. So it means the deregulator add the request new object. In here, and the user doesn't pass the request object
16:45
as the first argument. Yeah, this diagram showed how to work. This part inject the object as the first argument to create the request object and get
17:01
the request in the function. But the user doesn't pass the request object. So in our use case, sometimes we need to send the custom message, custom log to the external server.
17:22
So for example, we need the logger class to send the log in here. So this deregulator pass logger class as the first argument. And the callable side, the only user argument
17:44
is the URL. So it means same. But this type is correct. Precheck the overtyping. It's too long. So I will sort after these topics.
18:02
So pick up the important point, the callable concatenate, the first argument, the remote logger class with P. And this deregulator put the logger in here. And this function can get the logger class in here.
18:26
We kept the concatenate the flexible signature manipulation, enabling the adding the extra parameter to the function while preserving the original signature.
18:40
It's very cool that if we want to use a change to some other argument, it's a variable. Cool, yeah. Next, it's a new syntax from the Python 3.12. So precheck the example.
19:01
The first one, the previous Python, we have to import the type areas so they find a new overtype. But from the 3.12, it's the provided new reserved keyword type.
19:22
We don't need to import any new things. We can use the type to define the new type URL equal SDR. Also, in the genetics, we need the T. But we don't need to import that type to define the T. The new syntax that
19:42
provides the new feature, we can define T after the function name. It's very cool. Or let's apply this typing to our deregulator. So type the log func.
20:01
It means the log function, the callable type. Yeah, precheck that this double star, asterisk, asterisk p, it means the param spec. The p is the param spec. Also, the r is the return type r. Usually, we use the return type.
20:21
And the same, the callable concatenate, the remote log, and the p and r. And after the function name, we can use the same, double star p as the param spec. Yeah, raise your cap.
20:41
Provide the standards that way to the degree of type, parameter in the class, and the function. In this session, I show you only the function, but we can use the same way in the class. The precheck the PEP. OK, we can understand all the good things.
21:03
And we can apply all the things to our deregulator. It's too small, sorry. So I pick up the important lines, the defined response like with the protocol that
21:24
has the status code int, and to define the new callable type, the log func with a new syntax. And using the param spec with the double star p after the type function name, and to put the list
21:42
to enforce the keyword argument for the label. And we can use the p.args and the p.keywordargues. There also the function gets their log across the first argument, and put the log as the first argument
22:08
in the finish site. And we can call the sendlog method in this function. And we can return the response of the HTTP request, HTTPX.
22:30
This is finished, but I want to introduce the additional one. Typebar tuple, we can define the tuple typing.
22:46
I pick up the example from PEP. So this is a new one, typebar tuple. Usually we use the TS. TS is used by the asterisk.
23:03
It means unpack the tuple. So this argument, the TUP, is the tuple first T and the other tuple contents. But the return type changes order,
23:21
asterisk the TS and the T. It means pick up the first tuple contents to the end. So this example, the code is the 1, 2, 3. But the result is the 2, 3, 1.
23:44
It's very variable for the type checker and for the ID. Yeah, recap, the typebar tuple, it's good to define the tuple function.
24:05
Yeah, thank you very much. It's all. And I have already uploaded all this example code and this slide in the GitHub repo. Thank you very much.