The Story of Features Coming in Python 3.8 and Beyond
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Title of Series | ||
| Number of Parts | 118 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/44770 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
| |
| Keywords |
EuroPython 2019101 / 118
2
7
8
15
20
30
33
36
39
40
45
49
51
55
57
58
63
64
66
68
69
71
74
77
78
80
82
96
98
105
107
108
110
113
115
00:00
Beta functionPhysical systemPauli exclusion principleProjective planeOpen sourceNumberWordBlock (periodic table)Lecture/Conference
00:44
Parameter (computer programming)Regulärer Ausdruck <Textverarbeitung>Type theoryCommunications protocolPauli exclusion principleLocal GroupStatement (computer science)Operator (mathematics)Rule of inferencePersonal digital assistantSoftware developerCore dumpSheaf (mathematics)CodeProcess (computing)Formal languageCompact spaceSemantics (computer science)Complex (psychology)Transport Layer SecurityFormal grammarExpressionContext awarenessRight angleAuthorizationMereologyLevel (video gaming)Line (geometry)Statement (computer science)Hash functionInterpreter (computing)Operator (mathematics)Sign (mathematics)Rule of inferenceSoftware developerFlow separationPoint (geometry)MathematicsRevision controlBitImplementationCASE <Informatik>Software bugNumberLibrary (computing)Video game consoleEndliche ModelltheorieAdditionGoodness of fitElectronic mailing listSheaf (mathematics)Local ringCodeFunctional (mathematics)Different (Kate Ryan album)Module (mathematics)Social classVariable (mathematics)Real numberProcess (computing)Formal languagePauli exclusion principleMatching (graph theory)Semantics (computer science)Regulärer Ausdruck <Textverarbeitung>Sinc functionEmailoutputOrder (biology)Exception handlingWater vaporPattern languageWeb 2.0Core dumpForcing (mathematics)Pi1 (number)PlastikkarteComputer animation
09:43
Parameter (computer programming)Position operatorPauli exclusion principleFunction (mathematics)Positional notationString (computer science)ParsingPreprocessorNumbering schemeError messageConsistencyCommunications protocolLatent heatType theoryPersonal digital assistantCovering spaceFluid staticsParameter (computer programming)Communications protocolFunctional (mathematics)ImplementationElectronic signatureVariable (mathematics)MereologyArc (geometry)Extension (kinesiology)WordFlow separationCore dumpAttribute grammarProcess (computing)Operator (mathematics)CodeContext awarenessLatent heatReal numberArithmetic meanSoftware developerKeyboard shortcutPattern languagePower (physics)1 (number)CASE <Informatik>Module (mathematics)Single-precision floating-point formatVideoconferencingType theoryArray data structureLibrary (computing)Key (cryptography)MathematicsLimit (category theory)ParsingData structurePrice indexPay televisionElectronic mailing listInterior (topology)Position operatorSubsetTerm (mathematics)Formal grammarBitRepository (publishing)Shader <Informatik>Data dictionaryFormal languageOrder (biology)Social classString (computer science)Data typeTypprüfungEquals signMilitary baseAdditionPreprocessorRegular graphAuthorizationParsingNumbering schemeExpressionPauli exclusion principlePiOcean currentComputer animation
18:42
Pauli exclusion principleUsabilityComputer configurationTask (computing)Concurrency (computer science)Statement (computer science)Real numberSubsetType theoryMehrprozessorsystemProcess (computing)Interpreter (computing)TwitterInstallation artBeta functionIntegrated development environmentSoftware testingCycle (graph theory)InformationSinc functionModule (mathematics)Software developerProgrammer (hardware)Limit (category theory)Presentation of a groupCore dumpSocial classPoint (geometry)Object (grammar)Online helpCASE <Informatik>Software bugTraffic reportingType theoryElectronic mailing listMultiplication signRight angleFluid staticsOperator (mathematics)Run time (program lifecycle phase)Positional notationStatement (computer science)Task (computing)Concurrency (computer science)CodeFormal languageAreaException handlingSystem callSubsetWeightMathematicsBlock (periodic table)Video gameSingle-precision floating-point formatProjective planeRoutingData structureCoroutineUniversal product codeParallel computingMultiplicationLibrary (computing)Extension (kinesiology)Cartesian coordinate systemPhysical systemEvoluteProduct (business)FreezingBitWordVideoconferencingDifferent (Kate Ryan album)Computer programmingDistribution (mathematics)MereologyClique-widthLaptopRevision controlResultantPoisson-KlammerSquare numberInterior (topology)Interpreter (computing)Pauli exclusion principleMeta elementUsabilityRootMetaprogrammierungXML
27:40
ChainJava appletSimilarity (geometry)PlanningSpeech synthesisOperator (mathematics)Software developerFormal languagePauli exclusion principleParameter (computer programming)LinearizationAuthorizationMultiplication signPosition operatorAdditionDivision (mathematics)CASE <Informatik>MultiplicationEntire functionRevision controlMoment (mathematics)Uniform boundedness principleEmailLine (geometry)Electronic mailing listLecture/Conference
Transcript: English(auto-generated)
00:06
Hello everyone, my name is Andrei Blasovskaya a few words about me. I'm from st. Petersburg, Russia I'm the technical lead of PyCharm. I work for JetBrains here is a number of open source project or projects that I'm maintaining and I've also been contributing to the
00:25
typing peps for Python type system But today we are going to be talking about Python 3.8 as you may know it is going to be released In October this year and the second beta is actually available
00:41
You can go to the python.org website download it and play with the new features about which I will tell you today actually, there are Very good docs on what's new in Python 3.8 They go into details about all the new syntax all the new additions to the standard library
01:00
All the new stuff related to performance of the easy Python interpreter all the major bugs that were fixed there and so on and I am NOT going to be Retelling this document because I just don't see any sense in just reciting documentation Telling what's there and so on
01:20
so the best thing for you to do is just to go to We have this link and look into the features all the bits that you are interested in So this is not what we are going to be discussing today What we will be discussing is the story behind some of the features of 3.8 I will tell you their history how they appeared and what are the pros and cons of the new features of 3.8
01:44
Also, I will give you a small overview of some features that might come in the future version of Python beyond 3.8 Well, I said some features Well, yes some because there are just too many of them and I cannot tell you all about every single feature
02:01
So I've chosen some features that I like or I don't like for this talk But I'll do my best to and try to stay objective But I'm only half human half Vulcan. So, you know, it may not work out as I expected well, actually Spock was half
02:21
Well not forget it Okay, let's start with the new syntax because it's something that always catches your eye when you look into the new syntactic changes to your language Let's start with assignment expressions. Maybe the most famous feature of 3.8 Everybody is talking about it. It's the new wall or separator because because this assignment expression looks like a little bonus
02:47
Here's an example just to give some context I guess everyone is familiar with a regular expression module everyone Has written code like that only You usually put the assignment for the M much on a separate line and now it can be a more compact like that
03:05
All right. So this is what we are talking about But the initial pep was quite a bit different in the initial pep the syntax was that you had to use the required parenthesis around your Expression assignment expression and instead of this new assignment operator you would use actually the s
03:25
keyword So this was like the initial proposal also in the initial proposal there was this idea of sub local scopes meaning that these variables these names that you introduced via this Assignment expressions are local to the statement in which you have assigned them
03:43
For example in our previous case for the match the M name Would be available only in the if statement and not beyond after this if statement And since in Python we have only scopes only for functions and for classes and for modules
04:03
It required actually some name mangling in order to be able to put it into already existing infrastructure for holding names for local scopes of functions So the idea was that these names were somehow mangled input into the Locals dictionary so that you cannot access them directly after you get out of the scope for this
04:26
assignment expression After the pep was the draft of the pep was published into pie inside Python deaf mailing list It got a lot of attention of various developers including a lot of Python core developers And there were a number of changes proposed to this pep
04:43
Also, Guido van Rossum became interested in this pep and some of these additions were proposed by by him for example This wall separator appeared instead of s because s is already used in three different contexts in the context of importing things in the context of catching exception in the except clause and
05:03
also in the context of the with statement and since in every one of these cases s means a bit different thing The authors of the pep proposed to use this new operator for that this operator comes from ancient languages like Pascal and so on
05:22
Also, it was decided to drop the parenthesis at least in some cases and to introduce more Regular scoping rules without this name mangling thing So the names are still available after your statement with these assignments expressions is executed
05:40
Also since now parenthesis are not a required thing the pep authors had to deprecate the The you the usage of these assignment operator at the statement level. So you cannot write for example X Force operator to you cannot or you can only use your regular equals as
06:03
Expression for that because otherwise you would have had two different ways to do things Well for one with the most basic cases of using Python another thing that is That was required for this pep was scoping rules for comprehensions in
06:22
least comprehensions Comprehensions and any other type of comprehension the names introduced in the inside the four here are Actually local to this comprehension in Python 3 but for Assignment expressions it was decided to allow these names to be usable after the comprehension is
06:42
Executed in order to basically allow this pattern of usage where I want To use this assignment expression inside the any function So I want here to know if any of the lines in this list of lines Starts with the hash sign and for that I really often
07:03
Need to know the exact line that starts with the hash and these nice syntax allows us To basically grab this line and to use it further in the in your in your code So it resulted in a quite large pep with a lot of stuff going on and many of these things were quite
07:25
Controversial some people like the idea of assignment expression side people don't like it So there was this like dispute about whether it worth to be a part of Python or not and We don't decided that
07:41
It's still a good idea to accept this pep Even though the authors of the pep had to add a section about the importance of real code right in the beginning of the pep Stressing the importance of examples that are typically that typically occur in your code compared to
08:02
Some complicated examples you can come up with if you think how you can abuse this new operator But it's still helpful in many practical context like the one I the ones I I Show here for example to emulate something like do while that we don't have in Python yet Or think the thing I wanted always
08:22
To be able to capture the if part of a list comprehension and use it further in my comprehension expression It's I think it's useful thing So after accepting this pep Guido decided to step down from his role as bdfl I guess because he was too tired with this Controversy around this pep and also it's something it was something that he was already thinking about for example
08:46
He announces it at some picon us I guess it was in Montreal several years ago that he will eventually he would eventually step down from his role as bdfl And now after some discussions, we now have a new governance model for the Python language at the Python steering console
09:03
I wouldn't go won't go into much detail here because it's not the subject of my talk This is the pep number that described the process So to summarize there are some upsides of the new operator It makes it more complex to use assignments in some specific situations that very often occur in your code
09:22
Like the one with the regular expressions also the current implementation of this pep the current proposal is Has a relatively simple semantics. So if you don't any if you don't know anything about the assignment expressions It's still possible for you to understand what this card does
09:42
I guess at least it should be because the code is quite quite clear The meaning of this operator could be obvious from the context and so on. So it's not like super complicated But as some downsides you want I said that yes You can definitely come up with a very hacky example for which it will be completely
10:04
Obscure what what's going on? You won't be able to understand it without running the actual code. Yeah, it's the case Okay, and now we have basically two assignment operators in Python instead of one Well, we could have used the same assignment operator like the
10:21
single equal sign for that but it would lead to some issues with parsing and to some issues with assigning to attributes and to expressions with subscription indices and so on and if you want to know more about Python parsing and how this Operator was implemented and so on
10:42
I recommend you to watch a video of one of the talks at the current euro Python by Pablo Salgado Was one of the core developers this talk Dives deep into the structure of the Python parser the limitations of its grammar and what can be done and what cannot be done
11:00
The next thing also related to the new changes in the syntax positional only parameters It looks like that with the slash And the oops, sorry, the idea is as follows In the Python C API there were always functions that accepted only positional arguments
11:22
For example this power function the built-in from from the standard library You cannot pass your arguments by name, even though the documentation for this function Specifies the name the names of the arguments just because well is just how it works how it's implemented in the C API and
11:41
for Python we really didn't have a way to express it in on the in the Python language only in the as a part of the OC extension and people used this workaround to Have this star arcs argument and then they are assigned basically the parts of this Topple to specific local variables which
12:04
Worked but it was not ideal because it was hard to get What this function did just by looking at its signature and you had to look into the documentation or the implementation of the thing So now we had and have a new syntax where everything that is before this
12:23
Separator is actually a keyword. Oh, I'm sorry a positional only argument and everything after the slash is anything else Keyword or positional or only cured and so on? But why slash for positional only? Well, it's because in Python we already have this pep that describes
12:41
So-called argument argument clean DSL. It's a DSL that defines a preprocessor for writing C extension modules for C Python and In this DSL slash is used exactly for this case for separating positional only arguments like the ones in the function power From anything else and also the slash what was used I guess before in some documentation for built-in functions as well
13:06
So for historical reasons slash seemed like a good way to separate these parameters but wait, we already have some markers to separate or at least to Define keywords Parameters positional parameters and it's all it's usually the pattern to call them star arcs
13:26
Double star keyword arcs and so on. Why don't we why cannot we use just them for example single star for Positional on the parameters and double star for keywords on the parameters. Well, the reason is that in Python 3.3 or 3.5 we have now since Python 3.3 or 3.5
13:46
We now have this keyword on the arguments and for them We use single star as a delimiter of the parameter release So the overall scheme for parameters in Python 3.8 looks like that. You have positional only slash
14:05
keyword oppositional star single star and then keyword only arguments The reason for that is backwards compatibility We cannot just change things to make it to use star as a delimiter for positional only arguments because it's already
14:21
In use for keyword only but why is star used for keyword only arguments? That's a good question. Actually. The reasoning is described in the PEP that introduced keyword only arguments and Basically The explanation is there was this workaround how you can use a keyword on the arguments in
14:43
say Python 2 if you have this star ignore argument, and then you want any other argument after star ignore to be Keyword only you can check if ignore actually And Consumed some of the arguments that you pass to your function and
15:03
For example, if you pass key not as a key as a keyword argument But as a positional only it will be consumed by ignore and you can check if ignore is not empty and then rise For example erase for example a typer. So if you remove this word ignore you get this Single star that is used now in Python 3 for separating keyword only parameters from anything else
15:25
So to summarize We now have a consistent situation when we have syntax to you to define both positional only and keyword only arguments in Python functions and It's backwards compatible, which is always a good thing and the downside is yeah
15:43
It's well not consistent with star arcs and double star keyword arcs. But yeah, it's a compromise Let's talk a bit about stuff that is not related to new syntactic changes About new types in the standard library. There are several of them. I won't go into much detail about them
16:03
Just to mention briefly what they give you Protocol types give you a way to express basically duck typing in Python or if we Use the terms from type theory its structural typing as compared to nominal typing that we had in Python before Also, there are literal types that allow you to specify a type of a variable by restricting it to be of
16:28
to accept certain values of type int or string also, there are final types that allow you to for example for bit reassigning to a variable or You can define classes or functions that you cannot overwrite later
16:43
When heard from later and also there is type dicts that allows you to specify types for specific keys in your dictionaries What I wanted to tell you about in in this part is that where all these ideas about types come from Well, actually this process is a bit different from what I described previously
17:03
Yes, there are perhaps but the ideas actually come from the implementers of specific type checkers And there is a repository on github where people discuss these new ideas in issues in github issues So the process usually looks like that Some type checker. Let's say my pie come comes up with an idea of a new type
17:25
And usually these idea comes from some real-world Code base for which they had issues annotating this code base So there are some specific situations and they come up with the new idea of a type and put it into their own module like my pi extensions in
17:43
This case and later if this type is interesting useful other the authors of other type checkers like pi charm or pyra Also become interested in this and after some discussions these new types Moved to the typing extensions module, which is a pipette package and so on
18:03
but at least it is something that is Common for several type checkers and more and more type checkers start to support new types at this point And after that if some type appears to be a universally useful the authors of this extension come up with a pep draft and
18:22
Then the regular pep process takes place and after that things are being moved to the typing module To summarize the new additions now We have more cases for more types that cover more cases of the real world code bases Like the one at Dropbox where the my pi team actually actively annotates stuff with
18:46
with the help of my pi But the downsides are that the typing module is becoming larger and larger and it's hard to learn about Python types It's becoming a harder and also I would say that it's not always necessary to put stuff in the standard library
19:04
Especially if it's only used actually by a type checker with which is not a part of C Python distribution anyway Also, I can recommend you to watch to watch this video from this euro Python by vita schmid
19:21
about static typing beyond the basics, I guess in this talks There will be mentions of types. I just described So now let's talk a bit about stuff that isn't coming into Python in Python 3.8 Something that is currently being discussed or even implemented
19:40
But it didn't make it to the point when the Python the when the feature freeze occurred for Python 3.8 The first like cluster of things I wanted to mention are the new ideas about The evolution of typing in Python there are several proposals some of them are already pep drafts and
20:02
Most of them are about usability issues of the Python type system, you know Many people complain that the type system of Python is really messy Because it was something that wasn't there from the beginning and there were limitations if that syntactic limitations and others
20:22
so Now it's like you have to import a lot of stuff For example, you you have to import from typing the uppercase list just to express that you have like a list of int And there are a few peps that allow That propose to clean up the situation one of them proposes to use the newly introduced
20:40
dander class get item method to add actually this Square brackets operator for the built-in classes like lists or set or whatnot Instead of compared to for example, not introducing dander get item on the it's their classes meaning meta classes of
21:02
The actual lists the class type because other in this case every object in Python would have this Dander get item and so on. So this is dander class get item to overcome this limitation also, there are some ideas how to Make it easier to use by not requiring
21:23
Programmer to actually import things from the typing module So there is this idea since type hints can already be evaluated only statically via the new from future import import annotations There is this idea to switch all the syntax of Python type hints of Python typing to use type hints
21:44
That are evaluated only statically by static type checkers in that way It won't be necessary to actually import anything and you can just specify something that is only there for static type checkers and that doesn't affect your
22:00
For example performance of your runtime when you run your code in Python And also there are other ideas as well in this area Another area when where there are some possible changes coming up in the next releases is the area of asynchronous programming Async-await the idea is to introduce a call structured concurrency for asyncio tasks
22:23
The idea is simple when you run an asyncio task You don't really control it and it's hard to control it when it when it's going to be finished how you cache exceptions from this task and so on and so one of the core developers Nathaniel Smith
22:41
Introduced this library called trio where you can actually define so-called nurseries that spawn tasks and their scope They their scope of life is limited to this block where the nursery is active If anything bad happens to one of the tasks The nursery will wait for all the tasks to finish and then you'll have like a multi exception
23:05
that combines all the possible exceptions from all the tasks in the in the scope and you can Will be able to handle the situation in a clean way. This is Something that is comparable to the classical issue of the go-to statement that as you know
23:22
Consider is considered harmful But the title of the of Nathaniel's paper was the goal statement considered harmful Go is a statement of the language goal that allows you to spawn spawn coroutines in an uncontrolled manner So the idea is to have this structure like with wiles and eaves and so on that replaced the go-to
23:42
The idea is to have similar like structured patterns, but for launching asynchronous tasks And I would also recommend you to watch a talk from this euro Python bond by Lin root where she mentioned trio like nurseries
24:01
And There are also a couple of things I would like to tell you about performance One of the projects that I'm personally looking forward to is my Pisces It's something that could make your single core Python applications run faster by basically compiling them to see with a lot of calls to see extension API's
24:23
But the idea is a bit different from what Cython does in Cython You are basically adding more C types to convert your Python program Into a Cython program with just a different syntax Whereas in my Pisces the idea is that you limit yourself in your use of Python meta programming
24:46
monkey-patching and so on so your structure of your object behind of your objects become becomes more static and also it You are required to use type hints basically everywhere for your code So it's like a stricter more statically typed subset of Python
25:03
And if you restrict yourself to this subset Then my Pisces promises at least initially for some cases to give you like five X performance boost or something like that and it's actually already there in some production scenarios For example my Pisces as you might have guessed the name comes from the type checker my pi
25:25
and my Pisces is written by the members of the my pi team and They have created this project to make my pi run faster and this is the result they got like five X five X performance boost and It's already there in the current version of my pipe
25:40
So if you are using my pi you already have this kind of technology working on your laptop Also, there is the idea of sub interpreters, but I won't go into much detail here because today We had a keynote about Python performance by Victor Steiner that had a part about sub interpreters and in general
26:03
It was a very interesting talk that touched a lot of areas about Python performance. So I would really recommend you to watch His video if you haven't attended the keynote today So a bit about the release plans as you might know the Python 2 is going to be retired
26:22
on the 1st of January 2020 3.8 as I said will be released in a few months and 3.9 It's I guess it's not clear yet. At least maybe maybe this information is already outdated But there was an idea of switching to nine month release cycles
26:41
So Python 3.9 might appear sooner than you expect So in conclusion, I really recommend you to install Python 3.8 better try it by yourself try the new features see if you like them or not and Try to run your production code in some environment at least testing environment to see if there are any
27:06
Incompetibilities and please report anything you find to the Python bug tracker and Help with fixing bugs with with coming up with new ideas for the next Python releases and so on and also I should mention then That all the features of Python 3.8
27:20
I was talking about today are available in one of the better builds of PyCharm 2019.2 I guess this concludes my presentation. Thank you All right, we have
27:41
Time for a few questions So we have two sets of microphones over there. Please line up and fire away Hello, this is working. Okay, great. So I'm the author of argument clinic and you might be wondering why slash in the first place
28:01
I use slash for indicating positional linear arguments in argument clinic because this was a syntax proposed by Guido van Rossum Arguments and you may be wondering. Okay. Why did Guido propose it? And the answer is because it's kind of like multiplication and division operators
28:20
Like one is the opposite of the other that was really only his thoughts cool Great. Thank you Hi nice speech I wanted to ask you as a question about Doing all the plans
28:42
Will be the question operator Similar that we have in C sharp or Java There are pep drafts about that. Maybe even two that Talk about this chain of nullity checks But as far as I know, they are not pushed forward at the moment
29:04
Yeah, thank you Or the quarter developers at all thinking about The size of the syntax of Python Over Python 2 we have very few additions to the language
29:23
Whereas we're seeing more and more with the new versions Yeah, but Python 2 is actually hard. I teach Python and it's hard to get through the entire syntax of Python in a week Yeah, it's definitely the case that it's harder to learn Python 3 than Python 2
29:42
Because the language is just bigger, but I should mention that Python 2 is I guess 11 years older than Python 3 and Python 3 just lived a lot longer So it acquired more and more features if Python 2
30:01
Would have changed beyond Python 2.7, I guess there would be some new additions But yeah, yeah, I agree. That is the problem and this is one of the things I mentioned About typing even though it's not related to syntax. But yeah, I guess it's an issue Alright, let's thank Andrey again. Thank you