Type Annotations in Python 3: Whats, whys & wows!

Video in TIB AV-Portal: Type Annotations in Python 3: Whats, whys & wows!

Formal Metadata

Type Annotations in Python 3: Whats, whys & wows!
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Type Annotations in Python 3: Whats, whys & wows! [EuroPython 2017 - Talk - 2017-07-13 - Arengo] [Rimini, Italy] Python 3.5 introduced support for ""type hints"" (or annotations), which allows us to annotate our code with useful bits of information without affecting its runtime behavior. Type hints don't enforce a particular use case and can therefore be used for many purposes. In my talk, I will explain some of the use cases of type hints, and show how we can use them to e.g. make our code more secure or teach it new tricks. We will have a look at popular libraries that help us to use the power of type hints (e.g. mypy), but we will also see how we can build our own extensions on top of the type hint system. After the talk you should walk away with a solid (basic) understanding of type hints in Python and an idea of how they might be useful to you now or in the future. Typing and type hints in programming languages Short history and motivation of type hints in Python Basics of type hint syntax and semantics Use cases for type hints Type hints for code analysis: mypy and similar libraries Building new stuff with type hints in Python Summary, Outlook & Further Readin
Fluid statics Computer font Goodness of fit INTEGRAL Multiplication sign Network topology Projective plane Mathematical analysis Code Computer icon Formal language
Code INTEGRAL Mereology Computer programming Software bug Formal language Mathematics Strategy game Different (Kate Ryan album) Cuboid Information Error message Formal grammar Physical system Proof theory Block (periodic table) Cycle (graph theory) Gradient Fitness function Bit Unit testing Funktionalanalysis Variable (mathematics) Position operator Arithmetic mean Vector space Order (biology) Phase transition Website output Software testing Pattern language Right angle Hacker (term) Reading (process) Point (geometry) Identifiability Observational study Video game Letterpress printing Mathematical analysis Revision control Latent heat Goodness of fit Software Software testing Form (programming) Consistency Projective plane Code Line (geometry) Compiler Error message Software Personal digital assistant Network topology Formal grammar Statement (computer science)
Point (geometry) State of matter Code Multiplication sign Formal language Number Revision control Mathematics Goodness of fit Different (Kate Ryan album) Typprüfung Formal grammar Physical system Area Addition Software developer Theory Bit Funktionalanalysis Arithmetic mean Function (mathematics) Network topology Pauli exclusion principle Interpreter (computing) Communications protocol
Run time (program lifecycle phase) Electronic mailing list Sound effect Parameter (computer programming) Electronic mailing list Funktionalanalysis Parameter (computer programming) Variable (mathematics) Theory Computer programming Variable (mathematics) Element (mathematics) Sound effect Architecture Interpreter (computing) Network topology output Integer Arrow of time
Installation art Computer file Code Confidence interval Decision theory View (database) Drop (liquid) Data dictionary Mereology Computer programming Variable (mathematics) Formal language Pie chart Hypothesis Number Sound effect Revision control Architecture Goodness of fit Interpreter (computing) String (computer science) Personal digital assistant Core dump Integer Endliche Modelltheorie Social class Physical system Module (mathematics) Mapping Software developer Electronic mailing list Code Drop (liquid) Bit Funktionalanalysis Flow separation System call Function (mathematics) Order (biology) Interpreter (computing) Website Pattern language
Filter <Stochastik> Domain name Game controller Code Line (geometry) Multiplication sign Virtual machine Translation (relic) Data dictionary Theory Neuroinformatik Product (business) Software testing Data structure Domain name Interface (computing) Projective plane Electronic mailing list Code Price index Line (geometry) Word Drum memory Logic Personal digital assistant Network topology Order (biology) output Self-organization Software testing Pattern language Local ring Library (computing) Surjective function
Digital filter Scripting language Module (mathematics) Run time (program lifecycle phase) Code Parameter (computer programming) Mereology Number Element (mathematics) Bit rate String (computer science) Integer Endliche Modelltheorie Error message Fingerprint Social class Scripting language Module (mathematics) Uniqueness quantification Fitness function Electronic mailing list Code Instance (computer science) Funktionalanalysis Variable (mathematics) Function (mathematics) Order (biology) Software testing Library (computing)
Point (geometry) Group action Module (mathematics) Code Multiplication sign 1 (number) Virtual machine Instance (computer science) Mereology Product (business) Crash (computing) Software testing Damping Integer Endliche Modelltheorie Error message Social class Physical system Area Dependent and independent variables Information Block (periodic table) Code Attribute grammar Menu (computing) Bit Funktionalanalysis Variable (mathematics) Sequence Error message Integrated development environment Function (mathematics) Revision control Social class Lipschitz-Stetigkeit Quicksort Object (grammar) Routing Resultant Weight function Library (computing)
Scripting language Addition Execution unit Digital filter Code State of matter Projective plane Fitness function Electronic mailing list Attribute grammar Menu (computing) Parameter (computer programming) Instance (computer science) String (computer science) Social class Integer Damping Object (grammar) Glass float Resultant Library (computing) Physical system
Module (mathematics) Module (mathematics) Code Multiplication sign Code Parameter (computer programming) Funktionalanalysis Variable (mathematics) Electronic signature Conservation law Integer Endliche Modelltheorie Resultant
Area Pulse (signal processing) Information Computer file Computer file Code Directory service Client (computing) Funktionalanalysis Variable (mathematics) System call Arithmetic mean Integrated development environment Repository (publishing) Right angle Electric current Library (computing)
Addition Email Computer file Information Code Building Interface (computing) Computer file Code Mereology Funktionalanalysis Touch typing Resultant Social class Library (computing)
Tuple Standard deviation Group action Code INTEGRAL Source code Variable (mathematics) Software maintenance Mathematics Synchronization Different (Kate Ryan album) Endliche Modelltheorie Data conversion Pairwise comparison Library (computing) Social class Source code Computer virus Electric generator Building Computer file Electronic mailing list Parameter (computer programming) Bit Variable (mathematics) Pattern language Electric generator Resultant Point (geometry) Module (mathematics) Link (knot theory) Computer file Software developer Similarity (geometry) Branch (computer science) Generic programming Element (mathematics) Canonical ensemble Software testing Data structure Module (mathematics) Pairwise comparison Execution unit Information Weight Diffuser (automotive) Code Core dump Line (geometry) Software maintenance System call Personal digital assistant Function (mathematics) Network topology Pauli exclusion principle Revision control Interpreter (computing) Social class
Scaling (geometry) Computer file Code Token ring Computer file Projective plane Mathematical analysis Code Number Product (business) Medical imaging Repository (publishing) Personal digital assistant Network topology Order (biology) Repository (publishing) Software testing Diagram Quicksort Resultant
Onlinecommunity Letterpress printing Drop (liquid) Parameter (computer programming) Mereology Variable (mathematics) Number Goodness of fit Core dump Repository (publishing) Cuboid Proxy server Absolute value Physical system Theory of relativity Projective plane Physical law Funktionalanalysis Affine space Connected space Particle system Repository (publishing) Function (mathematics) Quadrilateral Booting
Dataflow Default (computer science) Run time (program lifecycle phase) Information Link (knot theory) Demo (music) State of matter Code Range (statistics) Control flow Design by contract Funktionalanalysis Parameter (computer programming) Variable (mathematics) Function (mathematics) Arrow of time Integer Position operator Typprüfung Physical system
Slide rule Game controller Code Multiplication sign Direction (geometry) Source code Virtual machine Computer programming Twitter Product (business) Frequency Fluid statics Goodness of fit Positional notation Different (Kate Ryan album) Software testing Integer Endliche Modelltheorie Extension (kinesiology) Mathematical optimization Form (programming) Physical system Theory of relativity Slide rule Information Moment (mathematics) Constructor (object-oriented programming) Projective plane Mathematical analysis Electronic mailing list Code Basis <Mathematik> Funktionalanalysis System call Abstract syntax tree Compiler Data model Order (biology) Website Right angle Quicksort Library (computing)
Personal digital assistant Integer Figurate number Number
so good morning everyone I'm pretty happy to be here and there was that the that there's such a large interested seems and type annotations and probably some of you already heard of some type of annotations no maybe even use them and so that's a good thing and because I've annotations are actually pretty old feature already and I wanted to give this talk because right now there's a lot of activity in this field and as a person we have worked a lot of static analysis and I'm interested in and everything that is connected to call quality found is an interesting and very good opportunity to actually have a look at that annotations again and see what happened in this ecosystems in the last 3 and 4 years and spoiler alert it's quite a lot that happens so I have tree learning time and goals on for you in this talk on 1st I want to explain to you why actually we need type annotations or we might need them and how they made their way into the pipe language and secondly I want to show you how we can actually use those that annotations and type and your own co-pays and for that we don't and have like an integration project we take a small corpus and actually tried to put type in there so and then I want to find show we also not only how we can use icons but how other people are using them in the wild so in public projects and want to see if they actually somebody else's you know it's doing something with that so finally that I want to see 1 to show you how what else you can do with the annotations if you're not interested in time and the OK
so the 1st question you ask is that is probably and why do we need to add annotations at all I mean after all our code is just running fine without them right so to answer that we and we just have a look at and how and when we find box when developing software so if we ask ourselves what's the 1st opportunity to find a bark when we have a new softer project and it's actually not the it starts even before we write the code in the design phase you know because we could just in a perfect world and think about what we want to build on like think about all the edge cases in all of the input variables that we might have a no problem and then I was that perfect specifications right the decoding something that has no boxes and of course this would be nice to you know on this almost never happens and actually there are some system that can help you want to write systems that right suffered is almost got but most of these but do not apply to many of the projects that we're working on so unfortunately we have to do something else the next opportunity to find a bug in our code is after we have written this of the cold and for example you could and have somebody else look at your code and tried to find blocks in the so called and actually this is a pretty good way of identifying box if you believe the empirical studies that exist so you're not doing is already should really start doing it and as an alternative instead of having a human read you could you could also feed into a computer program on the called static analyzer and have that study analyzes and try to find program and for Python also several solutions available for this that you should use and they can help you to identify some of the problems that can Huanuco and OK the last quantity doesn't apply to heighten but it applies to other languages so for example if you have C + + or last and you have a compiler that takes you cold and puts it into an actual executable except to the form and I'm quite doing so it also performs a lot of the so called type checking and using this type checking it can tell you before you run the program if there's actually some problem in there because there's some types that some don't fit together and as you know from your pattern experience on the pipe and the prevented sensory union because it allows you to almost run any code or any code you want as long as it corresponds to the piping grammar and I mean you probably all know the situation where you just make a small change in your code and then just go you run the program and it works fine in the beginning but at some point it crashed because you might have forgotten to include include like an import statement or some of your variables city using deep inside your program undefined and so this is a situation where like a compiler would really come in very handy because we catch those parts already before OK so if you can rely on that what can we do to final about snow program and what we can build a little exercise yard for a cold and and tested and that's what we're doing a lot of your unit testing functional testing integration testing with the goal of an actually running and as much as all quote as possible and and running into as many different situations possible in order to identify possible edge cases of things with the cold might break and this also best vectors so if you're not doing this already kind of house he recommended to start with that because can help to find many box in the program and so the last method that often is not discussed and but it is quite useful to find box is of course not find them but have the customer do that and that's what I call a for the so called the banana strategy because you know and if you like the shipping bananas you actually had taken off the the tree you when understood agreement on right and then you let them ripen at the customer site such as the solar cell 3 do the same just ship on really cold in a new customer kind of discovers a box for you and you go ahead and fix them and again I'm not saying this is a bad practice or something that happens a lot and but it's not always what we really want because we ideally want to deliver software the customer that is working as expected and not have him over discovered a box for us and the fighter community also for that and that's why I they decided to do something which is a bit like implementing a compiler for Python on by having a system that makes it possible to actually check on all programs for internal consistency before we run them so the type checker for pipe and if you want and that's what type printing is about the now the problem is
of course now already like billions of lines of Python code out in the wild and you know from the situation we had only to try to convince people to switch from Python 2 to Python tree that it's not so easy I want to get people to rewrite the whole cold in a new system so if you would just introduce like a new version of Python that would require everyone to like statically typed all of the cold and this would be a very good way good way to drive people away from might not have a fork in the pipe and maybe and so that's another strategy that the Python community pursued and instead that the so-called gradual typing and which is also pursued by other languages like typescript for example and which lets you instead of and having to esthetically annotate your whole coal-based let's see at type annotations of means to like small parts of your code you think it might be worth having them so and you see here we have some external code for example in grade would be annotated coding green it would be annotated code and then you can write your own code basically and you can also decide for which parts of the code base it actually makes sense to have this type annotations because maybe the code is really critical or maybe you really really want to be sure that there are no mistakes and all errors in that cold and to so this is nice but of course now we need a system always to also interface your own code with the external coat that might or might not have type annotations and later we will see how exactly we can do that and prices alright
so on typed annotations on by themselves are actually not a new feature and so I want to give you a short history of that from the 1st change that led to this current state and was introduction of so-called function annotations on which was done in 2006 so more than 10 years ago and it was formalized in the so called that which is a Python Enhancement Proposal and that's the formal ways of the city get new things into the Python interpreter of language in general and to step 3 1 0 7 introduce a new syntax that would allow you to have right to strike annotations in your Python code without so I'm giving any meaning to those annotations so basically what you could put in you could put in there whatever you wanted and the pipe but would just ignored and this feature landed in the pipe entry point of origin and 2008 and after that not much happened for 1 another community just wanted users to decide what they wanted to do with the future I think 1 of the users didn't do much with that but all anything from so until 14 that's why the
community again decided to and introduce some new and feature called tightens the step of 4 4 and is actually the fact that I gave meaning to the annotations are introduced into the 6 and there are also some other Pepsi afforded to 1 for a tree which can explain the fury and the ideas behind the type and so if you're interested in that you can check that out by just replacing and the number and inserting them into into the URA the defined below and so it is a new feature set landed in by converting tree . 5 and allows for the 1st time to really have a formalized system of using titans in the found in the language I could think and that might be the end of the story but actually know there's still some open and active developments going on here for example in 2016 16 there was another addition to the syntax of piping and be at that of 5 to 6 which introduced and a small change but a new possibility to also annotate a different type of thing that may be a variable because that's something that and the writers of the original Pepin 2006 didn't think about but which turned out to be quite useful if you want to do good type-checking so that's something that can be implemented just like last year in which landed in the new version of Python 2 . 6 and there's still some ongoing changes here for example that have 5 the for for which I find really exciting and which is about this called structural subtyping your protocols but which we unfortunately not gonna talk about that it about in this talk because goes a bit too far so as you can see it's a very active area down when new things are created and so it's at the state right now where you can actually start using
so now you probably say OK and stuff but the theory that's shown a sentence already so here's an example in all its glory and you might already spot some of the unusual things so here are some of the annotations in the cold place so you can see here for example we have a so-called argument annotation which annotates and the input value to our functions and if you that input value of course you also want to annotate what comes out of the functions of return value and insist on called return that annotation which is the human right and this is somebody's arrows in text and the final element is this new so called variable annotation which was introduced in by country . 6 which also allows us to annotate the variable inside a function body and you might ask is that why do we need that well it's actually useful because here what we have is that we have and we initialize the result variable with an empty list and for a type checker and this would be a problem because and it would only know OK this is a list of it would know what you would actually want to put into the lists and so this we can't edited by saying OK we want to have a list of integer values and that's the main motivation where this feature was added alright so they take a
step back now and actually think about what those tightens up have annotations and doing well if you just there would run the so called Titan and tree 6 on it actually would do not much nothing almost in our fight and Department of but it would just take the annotations and put them in a magic variable underscore underscore underscore annotations and then just go happily running by your program without any run-time effects also by putting these annotations are you do not change the value of your Python program per now
you also saw on maybe that we're importing this weird and what you there which is called typing and the motivation behind this typing model is that if we have a type checker we want to implement 1 and tighten the of course need a way to specify what types you actually have and and we have some good and types like integers and floats where you could do that and directly in the cold place but often we have types which a a bit more complicated so for example dictionaries that met the strings to integers and on lists of given another tight enough in the typing module helps us to specify the size but providing us and several classes in several ways to generalize these classes and adapt them to or all need when we're typing type annotating all program and you can see here 2 examples we have a mapping of a string to an end in the 4 me and we have this list of and inside the function body so we can now you probably asking yourself I'm OK if the patent a pattern that doesn't do anything with that annotations then on holiday actually getting used and the answer is and dating used by external tools so there several of them available for example on the most popular 1 right most probably my buy but there are also some ideas like pie chart that can use this type hands and these it's exam 2 is actually do the heavy lifting type checking so they go to the Python code look for the type annotations and then try to make sense of them and try to find problems in the cold and this decision of having that as an external tool and not as part of the poor language was a deliberate 1 on and I think it's a good compromise in order to have a system that is easy to evolve but I think eventually and a lot of the functionality might also land in the core Python interpreter or at least as a tool that is shipped with fight but right now we have to install an external tool for it the and now we can look at
1 of these tools and that's so as a call my prior the and Michael was written by you got a little side or I probably horribly mispronounces his name right now who is as far as I know working at now and develop many of the ideas for my by doing his PhD thesis and the the good news is that my prior strongly pushed by team drop works under the lead of the defined Ross which 1 should give you some confidence in the stability on like that leaves the active development of the 2 it's pretty easy to install you can download the there and then just run it on the files which were going to do in a minute and view light side also included their website which and you can see is quite active and they have like the releases almost 1 release 4 months and as you see from the version number to the still-evolving but it's already function enough so that you can actually use OK so the if you
want to use that we need of course in example code base and I choose a project and I did for the town and open so voted for the drums cybersecurity organization it's called or or is a German translation of the word bloom and so you might have heard of Bloom filters so what the libraries doing it provides such a long filter and what does it mean for the dual what it's and it's a so called probabilistic data structure to which you can put in some values and then you can later asked that the bloom filter have you seen this value or not and and if you have put in the value to bloom for that would always remember and always say yes I have seen the value but what can happen is also that it remember something so you know it might answer yes I've seen that with the value that you actually input put in the and this might not seem terribly useful but it's actually quite neat because we found that this is all we have lots of indicators on for example domain names and corresponding to domains that like control now they're all download mother onto computers and we want to check those domains and on the local machine for example so what we can do is to put all those values like a hundred million into a bloom filter which is then about depending on the probability that we choose of misremembering about like a 100 megabyte last week and discovered that the machine and asked if it every time have you seen this domain or not so pretty useful on library very small coal-based and we're gonna try not to add annotations to that and see what that goes so I choose a code based and as it
because it's really contact it's less than 200 lines of code but it actually does something useful so which I always find better than using like a toy example just to show how this stuff works it's actually something that people use in production and the nice thing about it also that it doesn't have any external dependencies beyond the standard library so you don't have to and type annotate all interface with any external code again this is good because it makes it easier and I don't want to give you the impression that it's always that easy to annotate code so of course if you have large projects you will need to put in some more effort to to type annotations but in order to get started and I think it's a good project to just see how it works and in theory but you can also actually do that logical place of course and it does not have many exotic types so most of the types of reusing and their a and saw simple compositions of pollutants like lists of dictionaries so it's not too hard to understand what's going on hope and last thing is that we keep that cope with compatible with pipe to 1 Pattern tree because some of our customers they can't use pipe treated so it's also interesting cases the type annotations can actually work for a type of pipe and to code base and in order to test that we need we
need a script of course so we have like and an example script here which imports are library and the imports that it creates an instance of this bloom filter class which we initialize with 2 on and values so normally it expects them value and which gives the number of the elements that are in the future and probability P which gives a false probability rate so the and the probability that the fit that would written this remember a value that we didn't actually put in there and the number of elements of be an integer of course the probability should be afloat and you can see here we instead put in a flawed and ends in the wrong place so this is something that would blow so to say in runtime and then and secondly we want to check added value to the furniture and again we have problem here because our filter expects the array of bytes and what we put in here is a unique code string so this again would be an error and lastly we want to do something with the fingerprint of a given value that the filter the returns and them what dysfunction actually transfers a list of integers but here our user kind of assume that it would be a string and he tries to watch addressed to add another string to that which of course again is a problem and now let's see if we can and get my pride to find these problems so
in order to X type can store base would go to the called function by function and then try to add and annotations the arguments and to return types so on if possible if needed we also might be might add annotations to other variables inside the function bodies or inside the module and to have on my PPI actually make sense of them if they're not used as a part of of function argument or return value and to do that we need to import of course a necessary types from the typing model and then and if we have done that we can just run my pi and try to make it happen so to say
all right and the result of that is shown here so you can see on that we added some type annotations to our in functions so we specify that n should be an integer that he should be a float value and we have a function also to read a bloom filter from fire and which of course expect something that you can read from and is also a way to express that using the typing model which is despite all class here and so what we're going to be telling the weight function that we are expecting an object well which is readable and which transpired that the function continues to read into the in the Bloom filter ones and finally this contains function which checks the value the values in the filter expects of course and a byte sequence and so by area and returns a boolean value which we also annotated OK and after doing that we can just run my prior and the shown here so you can see on the left I specified as an environment variable which is called myPipette which we need here because my product actually has its own sort of so if you want to check some external code we need to tell my pride where it can find that cold and in my case my Python libraries hinder lips folder and then I run my prior I also which pipe were to use in that case we want to use by country . because you don't use any variable annotation so we don't need 3 6 and not running anywhere I want to machine and then be tailored to our fight everyone to check in this case for test of P 1 and hopefully we would get an out and all the information about the mistakes that are in that fight but and to my surprise logo although in stead was like really weird cryptic error message which really made me solve that action understand anything about tightens at all so is really like when I do this OK and this is easy I get annotated stuff and and at what why by it uh but this is this should working up and so was like digging in the documentation and try to get some dividing and finally a found out the reason why this is not working and the reasons actually that I
broke my point with that and the problem here was that in the room for the class of variable which is called by its and which my prior and somehow mistakes for the bits and bytes type so when it tried to initialize that class here are my postings OK bytes is the very bottom but this is not the valid time so I'm going to show an error and then just crashes so as you can see it's still not perfect but on the other hand as they always like to the of your system is only as good as you most stupid user and probably I kind of knowledge that far from my part by naming my variables as the just like that time but a good news is the guys there were developing lapidary also so they are and I posted this issue here on Monday I got route the response to that and like 40 minutes and and lastly yesterday morning it was already fixed and merged into the master so as you can see their really like very very active and they're really takes the bacon back reports race you so if you have a block on that you encounter when running my by encourage you to just go to the gift of issue tracker and posted then they go and they would definitely help you are resulted in as fast as possible so
I'm having a fixed quick work around by the way was
to just rename I and variable with an underscore and then it didn't complain anymore
so the results of that and we can run on
our script again and the amazingly
and as expected the auditory problems that we put in here so you can see on when we initialize the Bloom filter it's all this is incompatible type I'm expecting a floaty it as a an and here and but I get a float and is actually also narrow for the 2nd method for don't show and then we try to add value to the fit of the wrong type and there might be also complex at all you should and should put in a bite objects but might you go strength and finally it's it's all and if you tried to have at a string to a list of integers and is also not possible so this is pretty pretty cool I find I mean it might not seem like a big thing but you have to realize that just by adding that if you asked methods of additional code to our project we made really much much harder for user for users to use our code the wrong way so maybe we make it much easier to like have a good state in the system and to avoid problems when using that library as a user but also of course make the cold better for ourselves in our so knowledge could be the end of the
talk and we could call home but if you just push that to master and it wouldn't take long for customer complaints the role and because as that some of our customers still rely on Python 2 and since type annotations are not supported by Python 2 and adding them to the code base would just break it the so that's bad news on but can we do something about that yeah actually can and because my prioritize conservative on a 2nd approach for adding this type annotation so-called ways which are called and type comments so type comment is found similar to a type annotation in the sense that we specify function arguments and return types as well as variables but instead of doing that in the cold we do it as a common so
if you would again do the same annotation as before Oracle on only this time using comments on the result would look like this so you can maybe I hope you can see them and so we're 1 comment after the init function which starts with this magic work type in the column which does my by the this is the type comment and then afterwards it contains the signature of the function so it's says that we have an integer value of the value and which are nothing and you can do that for all of your functions again as before and the nice thing about is is you don't have to use the new type annotations and that's what would you still have to do on this as you can see above imports the relevant types from the typing module which is possible at all because on the typing model has been backported button to 7 so you can just install that can be opted for example OK I know if I'm
right so I actually know because there's also and often cult that we just can't changing or maybe because of the because upstream somewhere we don't have like a right right so it's an external library that we want to use so there really out of luck pulse of cold common with type comments and with inline annotations on but luckily again this
approach that we can use for us in areas like this which is called a stop file and I've shown examples of stuff was so here we have our and director we used to call plots of or in it the PY the filter the PY which contains a call that we soluble and to some helper functions and what you can see here that we also put some new fires in there which are ending with few white I and these few i are of course a stop files that we're talking about and my client when going to call based would also try to find such stuff was in his own search and passed for that so look for example the current directory and in some the search prosody give it is my my variable environment variables and also in another place which called type show which is get help repository that contains many type information for for example built into Python functions and altered her body libraries soul and the thing is if my Pi finds a few why I file it will only load information from there and will completely ignore the corresponding QY and this means we can instead of like changing our coat on just at this few Wi-Fi few where I find that the type annotations there so
again it's the same on the same operation as before we just start with or cold and then we elect which would but now we in in addition to adding the annotations we also remove any actual code is doing stuff and so you could have probably think about this as like riding a header file for pipe and we know if elected con concept the idea but it's very similar to ever see your C + + had phi we specify the type information and have an actual phi that implements the interface of the city and the
result is looking like this so you have here again on our class with 2 different functions and you can see that we added our type annotations but you can also see that instead of having the function but written out we just put this ellipses this is treated adults which tells my private this OK there's a function body coming afterwards but it's not important so we don't put it there and so this is a very complex very nice way to specify the typing information for code that we can't actually touch and we can have the set stored in various places and have been my pride look for information when we try to change and to check code that actually uses or library for example so dysentery approaches that
showed you know you might wonder OK which 1 actually the best holiday compare and in practice and to get an idea that we don't push the street new branches to get up and we have a travesty I integration there which automatically automatically tests or cold against different PPI convergence and as you can see from the results are here so for that and if you look at the last element so we have all our stuff files in their and our code is still compiling or running fine which is not surprising because after all we didn't change any of the Python code we just added some new files that the pipe and the broader doesn't care about so fo for the interpreter interpretability something exists and but for the other 2 cases where we use in land type and so we use common to type and on we actually get error and if we go on look at the DTDs of that you can see here and the results for the in line for commented on built for different PPI conversions so we can see that if you look at the left column and in London for pipe 7 of course face because the similar type annotations and of and 2007 in the same of course for pipelined Piper 5 3 1 and here this failure for pipe entry point 2 and 4 pi pi tree is more surprising because it seemed exclamation mark instead of an X. which means that something went wrong during the set of of the test not doing the action running and what went wrong there is that we try to import the typing what but unfortunately the typing model requires the type entry point trees so quite DuPont to kind imported and neither can type I and that's why those 2 tests of failing and the same is by the basically the reason why the commented patterns are failing here because also but we still have we only have comments in our court but if you remember we still need to import the typing model so we still have a dependency on that so this is why the code is not running on pipe and she . 2 a pipeline and actually this is not the mutual and because I follow the work around for this which is maybe a bit ugly but it works and so you can if you follow this link here actually also make the comment tightens the work pipe and you want to apply poetry right so found so which approach of use some it is look at the on-line approach that's kind of course the right way to use that and it's a weight type and subtype annotations are intended to be used and the advantage is that it's is easy to read and that the cold and the hands are kept in 1 place and the drawback is of course that you could only be compatible with 5 and 3 . 3 or diffuse variable annotations of partnership on succeed and get out if using typed commands and is good because it keeps a call competitive Python 2 . 7 and but it also uses and also used by of Annotations but it might be a bit ugly and it also still requires importing the type what you which there's work around for the the final producing the stuff I is a nice because doesn't modify any of the original source all and allows it to use always the latest features that are available in my prior so on that about disadvantage of course that you can and that you have to right now and so 1 file on 2 so you would kind of duplicate you're maintenance efforts and you will always have to take care to keep those 2 fights in sync with each other which is not always easy so right of course is much more to the
on the type and send this and we just like have a look at a very simple example and if you want to learn more about that the recommended to check out the documentation of the piping module which contains information about how you have for example can create types for your classes or class Europe's I you can use so called generic so think about a list that contains a different element of a given type on like a new structure that you invent and that use Nicole so the typing model provides support for all of these use cases and many more like generators as incomes things and so it's really a very well documented and and can own petard to type most of the variables that you will find a new everyday Python code I would say so
probably noisy OK but this is nice but is anybody actually using this already and I didn't know that so I checked and what I did for that is to download from the 1 the top 1000 repositories patent was stories from get up which you can easily do 3 truly API and then you can the code of that and check it for inline annotation type commons and stops so the code and the analysis can be fun and get up I just want to show you the results here so what you see here on
the left is the on an image showing you this 1 thousand projects that analyze so those are really the most popular pipe projects and get up and the coral code that represents the number of type and sort type annotations that are found in each project and you can see here on the bottom there's a scale so it goes from 0 Titans obviously to about like 10 thousand and so if you look at the number of projects that actually use a type cancer can see that from these 1 thousand projects 100 projects which have at least 1 type annotation the cold and so you might say OK this is like and homeopathic dose of type and so and maybe it would be more useful to see the project x to use many of them and if you go on for example you check of number of projects that have at least can type in called base and it's going down to 50 tree already and if you check for products that have 100 and then it goes down again to about 24 so you can see that on popular projects and get up there already starting to use type annotations and some of them are really embracing them but others just like maybe experimenting with them more like using them in some test cases and in order to see which kind of type and so using you can also have a look at the right diagrams here which shows you the usage of inline tokens type commons and PY files and if you're looking for some
great examples of of some type annotations of supply pens and particle places we can have a look at these repositories so and the first one is no surprise of course because it's my pride project so I I would expect them to use items if they're like writing that OK and the 2nd 1 item on you might know that if you wear them have like a connection to pick a school or a person because I think they use that as their main discussion system calls live and it's and online communities system which was also acquired by drop books on which might explain what using tightens a lot so that's a 2nd project and I mean this to you could maybe this comic is because of its OK work again part of drop box and maybe they're forced to use tiepins maybe don't like that you know the law and the the FIR predicts something that you probably used a lot and which assisting argumentation generator and I wasn't to surprised that they're using type and but they have a lot of them in the core principle for those wasn't an affine and they really made very well use of 10 so if you're looking for a corpus of good examples about how to use type and some really large project you should level look at that and afterward system you can see the number of type and the project is going down rapidly there some other things here which is I think this your H U R H is radio pecking tool and your some home automation tools Europe proxy and some some ABS as related stuff so overall you can see that like of the 1 thousand 500 posters that we looked at the least 10 that I would say on use type and the type annotations in a serious way yeah alright
so I'm almost true and the last thing I want to show you is on that you can actually use type annotations for other things than type printing because you if you remember I said the type annotations and this was introduced into those 6 and it wasn't tied to any extra use case so it was deliberately left open so that users could just use that on additional syntax to do stuff and now if you look again at a simple example of a function of type annotations like annotations in this case and you can
ask is that what is happening actually when we run that and as a set before what the pipe and the prejudices that like creates a special variable called underscore underscore annotations where puts everything that you put after the column here after the the arrow so the state so that makes it really easy to access that information and runtime and it allows you to them and that stuff on top of that and
as an example and again I don't show you much code here if you want them to see how it's implemented you can look at the ghetto just which is the link lot and so what I did as in as a small demo is to build a run-time type-checking using this annotations and it works like this you have a function here which in this case takes 3 arguments and we want to make sure that the 1st argument is an integer which is in the range between 0 and 100 the 2nd 1 is the flow which is between 2 and 20 and 40 and offered 1 it is again in between 70 and 80 which also has a default value and they also say that OK the return value of the function should be a positive flow value and is so-called contract because you specifying not in the body of the function what and the values should look like but already in the definitions of the arguments and then you have like a system that enforces that for you so if you call the function of an argument that doesn't correspond to the contract that you have on the mn this decorated you just called check would complain about it and this 1 is actually quite easy to but with this type annotation on the syntax and fightin and it took on less than 100 hours to implement that and I didn't know what the people should put it here because I'm not sure that's a good idea because and the thing is if you use it in this way you would of course on break and the type of painting with you know because and tools like my pride expected and the type annotations to make sense so if you put something like this year which might doesn't understand and it would be a problem so as that's why I'm saying you can I use the stuff in that way but you shouldn't you should
think about where OK to summarize and I hope I could show you the type and type ending in Python actually works and makes a call more of those for yourself and for your other people so that he can already use it now even if you have tightened to called basis and that annotations can do more than type and and if you want them to all right so that's basically it you can get the
slides and slides here you can find me on Twitter or competent humor and moment to take your questions thank you and then few who questions thank you for all the other than the last thing in the future is not aware of all using the type of publications for a optimizing by collective the compiler the and yet I think and there was definitely benefit ideas that are going in this direction i am I think the Piper product they considered this but this that that like the order sources of information they can use to do that optimizations on but of course like having this type annotations in the cold makes it much easier to do on like specializations of function so you could use a definitely to make pipe quote from an optimized where we have like a virtual machine that is running that is compiling into something else I mean site and this basically doing this by using a kind of different annotations and take so yeah it's definitely possible more questions Ch good and the thank you so this suggests also already the this to correct typing that human relation to the testing the of herself and then correctly or in the this is can also be used do the and so the idea of the the type checker is of course that you don't need to write any additional code to to check your go places to see and you can of course to that but this was mostly an example on where I want to show how you can use the type checking and a whole how users of your library could use the type checking in order to make sure that their code is using your call correctly and you can of course like right here on the test but it's not really necessary in order to benefit from the type checking information but it's possible the hi and you shall this they information on their annotation and the function in it is close to the formation of and data models and yet possible it should I don't remember where exactly the sort of thing it's also inside the like notations of it but we have to look that up but yeah and you can also cost get those from the annotations in EAST which is how I found them in the piping project on GitHub so just didn't run the colleges confided to an AC using the as the model and then look for the annotations in the syntax tree suppose ways of possible time so I know it's a little to the type of thing is this small multiple types is that it's all it's In the yeah there's and I didn't show that but the typing modular and actually has support from many of the things that you would need in a typing system on including so-called union type so union type would basically allow you to say OK this label it could be either a list of integers or a list of or maybe something else so you can do that and ensure that but it's possible now this talk there is also the reason why things using also annotations in the direction also by using an extension for things that who put your your comments like if you use my comments that it will search for them in would be OK yeah of course that's a good reason thanks for pointing that out now the last time I tried to use some sort of time periods and so I don't know how my mind works internally and I've read and written some static analysis tools myself so normally what you do is you try to control a constructor might call program control program on and in all the convert court to like singer and as is a form and then do something with that so I'm not actually sure how they do that but yet is that it is some type feelings involved so and what show you the code examples was that and I can just go back
here and you see that we have like these definitions you often of P and and the and there we just put like literally in there and my pi can of course and figure out that the value in there has a type of integer for example so it can do some of that stuff doesn't do I think too much of it but for simple use cases that can actually inferior types occasionally finally question but I'm looking use already using type it also includes an introduction who buys this is the beginning the year and it was the number that was not the name again and the other was I thought it would