Writing Good Python
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 130 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/50001 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 20202 / 130
2
4
7
8
13
16
21
23
25
26
27
30
33
36
39
46
50
53
54
56
60
61
62
65
68
73
82
85
86
95
100
101
102
106
108
109
113
118
119
120
125
00:00
Goodness of fitRepresentation (politics)Lattice (order)Meeting/Interview
00:30
MereologyRight angleVirtual machinePhysical systemSoftware developerLibrary (computing)MassElectronic mailing listWeb 2.0Software frameworkComputer animation
00:53
SoftwareSoftware developerConsistencyMIDIPunched cardDrum memoryPauli exclusion principleStandard deviationConfiguration spaceError messageCodeStudent's t-testPoint (geometry)PlanningSoftware engineeringElectronic program guideIntegrated development environmentInformationConsistencySoftware bugWeb pageMereologyJava appletBlock (periodic table)Parameter (computer programming)Error messageCASE <Informatik>Standard deviationSocial classElectronic mailing listFormal languageFlagCalculationHierarchyCodePhysical systemSoftware developerMathematicsPauli exclusion principleMultiplication signInverse elementSoftware maintenanceVirtual machineType theoryTouchscreenGreatest elementExterior algebraRepository (publishing)Functional (mathematics)Code refactoringLibrary (computing)Inheritance (object-oriented programming)Message passingBridging (networking)System callIdentity managementOrder (biology)Self-organizationCausalityDivisorBit rateOrbitRevision controlCondition number
05:38
Pauli exclusion principleStandard deviationWebsiteRow (database)Computer fileComputer animation
06:00
CodeConsistencyError messageIntegerLibrary (computing)Letterpress printing2 (number)CalculationElectric generatorCodeParameter (computer programming)Software developerDisk read-and-write headStandard deviationString (computer science)Statement (computer science)Mereology
07:14
Fluid staticsPauli exclusion principleOverhead (computing)Run time (program lifecycle phase)TupleModule (mathematics)Electronic mailing listType theoryModule (mathematics)Type theoryElectronic mailing listRun time (program lifecycle phase)Parameter (computer programming)Internet service providerTupleFluid staticsPauli exclusion principleDampingString (computer science)Video game consoleSign (mathematics)Arrow of timeOperator (mathematics)Standard deviationSocial classComputer configurationMereologyTape driveGreatest elementDisk read-and-write headMusical ensembleKey (cryptography)Order (biology)
09:37
ConsistencyCodeNegative numberCore dumpData miningParadoxInformation security
09:58
CodeInformation securityMachine codeUniformer RaumRepository (publishing)CodeInformation securityElectronic mailing listData compressionNetwork topologySoftware developerSource codeDemoscene
10:29
WritingFormal languageElectronic mailing listVideo gameSoftware developerConfiguration spaceHookingLibrary (computing)CASE <Informatik>Electronic mailing listINTEGRALCodeProjective planeSampling (statistics)Computer fileBranch (computer science)Revision controlMultiplication signRootOrder (biology)HoaxConcordance (publishing)MereologyData storage deviceSoftware testingPlanningCore dumpNormal (geometry)XML
12:55
Error messageString (computer science)IntegerStandard deviationConsistencyConfiguration spaceProjective planeRootComputer fileFunctional (mathematics)Type theoryCodeCoefficient of determinationInformation securityInheritance (object-oriented programming)CAN busSource code
15:35
Reverse engineeringGoodness of fitWebsiteRight angleMessage passingWhiteboardNeuroinformatikWritingVideo game consoleElectronic mailing listCodeProgrammer (hardware)Computer animation
16:12
Configuration spaceLibrary (computing)Self-organizationLine (geometry)Rule of inferenceComputer fileException handlingMultiplication signEndliche ModelltheorieLie groupQuantum stateMeeting/Interview
Transcript: English(auto-generated)
00:06
So Prashant is going to be talking about writing good Python. That's really cool. Okay, so thank you for joining. I didn't ask you, is this your first EuroPython? Yeah, it's my first EuroPython,
00:20
but I already presented it by con limerick, so. Okay, okay, welcome. Thank you for representing. And yeah, good luck with your talk. Okay, so let's start this talk on writing good Python. But Python is already great. It is readable. It has massive ecosystem of libraries, frameworks and tools.
00:42
It is used in machine learning, data analysis, scrapping, web development, IOT, et cetera. It has a vibrant community and the list goes on. But I'm going to talk about the writing part of it. About me, my name is Prashant.
01:01
I'm currently a software engineer at HubSpot Dublin. The idea of this talk comes from my experience working at a small startup or a small company, but fortunate enough to be working with big clients. Some of them are amongst fortune 500. We were writing automation systems in Python. So when you are writing code for a process,
01:20
which is done by a human in such large organizations, you have to be extra careful because machines don't double check. They do what they're told to do. Like this scenario and in general, our employers expect us to write high quality code. So what is high quality code? So first thing is it should be defectless.
01:42
And second thing is it should be maintainable. Maintainable can be roughly defined as inverse of amount of time it takes a developer to make a change and risk that change will break something. It cannot be strictly defined. It is also just by the readability coupling and consistency of the system.
02:02
Today, I'm going to deep dive into writing Python code with consistency. So a consistent code follow the style. It don't have any hidden surprises. A style guide is mostly cosmetic, but it helps us in avoiding some common logical mistakes and make our system more maintainable.
02:23
So let's start with an example of bad or low quality code. So this is a highly opinionated and simple example of a hierarchy calculator. Some of you might have seen something worse. Some of you might be okay with it. And for some of you, this might be a perfect code.
02:41
But for me, I don't want to be a developer who is maintaining this type of code base. There are some obvious red flags such as inconsistent naming convention. We can see the name of classes in small cases. We can see that in some places, the method arguments are in block letters, et cetera. There are unused imports.
03:01
And in the init function, you can also see a dangling semicolon. Maybe the intern working on it was a Java developer. Apart from this, there are some obscure bugs such as on the second part of the page. In the except block, you can see a user this dot maxint. So this variable is no longer available in Python 3.
03:24
So is there a way that we can catch these bugs and inconsistency beforehand? So Python is a language that comes with batteries included. So it comes with a style guide written by none other than the creator of language itself.
03:42
It is known as PEP8. PEP stands for Python Enhancement Proposals. So these are designed documents for Python community, which provides information about new features of Python environment or its processes. There is another style guide for Python docstrings known as PEP 257.
04:02
So now we have PEP8, but how can we enforce this in our code? So Pylint is a library which can help us in this. So Pylint can help in enforcing coding standards. It can detect errors. It can suggest refactoring and also very customizable.
04:21
You can look more about it on its webpage, which is mentioned here. So let's run Pylint on our previous example. You can see Pylint is able to detect so many errors ranging from unnecessary semicolon to unused variable or missing function or method docstring.
04:42
It is also given a score to our code. It is giving us 1.62 out of 10. So this code is far from perfect. You can see list of all checks done by Pylint in the mentioned GitHub repository.
05:01
There are hundreds of checks that can be done before and so to be helpful in code reviews. There are also other alternatives to enforce Pylint standards such as flakeit and pyflakes. So we have updated our code and fixed all error messages.
05:22
You can see at the bottom of screen that Pylint has rated our code 10 on 10. This might not look impressive, but when you are working in a team of developers, this kind of tool is very handy. So I mentioned something named pep257.
05:42
So we have seen that Pylint is able to detect empty or no docstrings, but sometimes we want to enforce more standards for Python docstrings. pep257 can help us in that.
06:01
To enforce pep257 standards, we can use a library known as pydocstyle. So as you can see that Pylint was giving our code a 10 on 10 rating, but pydocstyle is still complaining. There is further room for improvement. So if we use pep257,
06:23
and not only make our code more consistent, but we can also use another libraries to automatically generate code such as docsion, pydoc or Sphinx. So everything looks fine till now,
06:42
but can we make our code more consistent? So let's see another example. So here we are trying to like use a calculator. So in the second print statement, you can see we are trying to add an integer to a string, which will throw an error
07:00
because it is not permitted in Python. So is there a way that we can give a heads up to a developer that what kind of arguments does the method expects? So there is a library known as mypy, which can provide optional static typing for Python.
07:21
It is based on pep 484 standards. And the best thing is that there is no runtime over it of using it. So I have type annotated one of the methods from the previous example. So to do this, you can see that I've mentioned the type
07:42
followed by a colon for arguments. And for the return type, it is followed by arrow sign made using a hyphen and a greater than sign. And in the bottom, in the console screen, you can see that when I run mypy, it will, it can detect that we are trying to pass string
08:03
where a float was expected. So we can see that it is pretty easy to type annotate primitives, but what about other types such as list or like because in Python, you can also functions. So how to type in that?
08:23
Python has a module name typing, which has many classes, such as tuple callable list to type in those corresponding types. You can also create your own types. I mentioned that mypy don't have any runtime over it,
08:46
but sometimes this is very tricky. For example, you can see on the right part of the slide, there is example where we are trying to type annotate a method argument named obj by class A, which is imported from another module.
09:01
So sometimes like there are some operations which are performed as soon as we are trying to import a module. So this import for only the sake of typing is costly. Sometimes we want to avoid that. So there is a special constant named type checking
09:21
in the typing module, which is true for third-party static type checkers, but it is false at runtime. So if you use that variable, then during execution, this module is never imported. So we have pylint, pydoc style, and mypy under our belt.
09:46
Can we make our code more consistent? So there are two more libraries, bandit and black, which can help us in achieving the consistency which we are striving for. So bandit can detect security issues in the Python code.
10:06
There is a long list of all the checks which it can do to improve the security of your code. You can see the list of examples in the mentioned GitHub repository. Black is a source code formatter.
10:22
It is very useful for source code uniformity across the team. So we discussed lots of tools here, but how can we integrate them into our development life cycle? So according to me, they should be used
10:41
while writing your code before committing and after pushing. So you can use ID integration of these tools to highlight mistakes while writing your code. You can use get pre-commit hooks to run these tools before committing, and you can use them in your build pipeline
11:01
so that they run alongside test cases. So I'm going to discuss more about how to use these tools as a pre-commit hooks. So instead of manually managing them, we can use a Python library known as pre-commit to use pre-configured git hooks.
11:23
So all the configuration is done using a YAML file. So you can read more about the project on its website, and you can see all the list of available hooks. So this is a sample configuration file to use pre-commit.
11:45
Fortunately, all the tools which we discussed are available to be integrated as pre-commit hooks. This configuration will seem obscure to you, but after like going through a documentation, you can get it like it is very easy.
12:02
We are mentioning the repo, which is hosting our hook. The rev is the revision or the branch which pre-commit should look for, and id is the id of that hook. So if you put this YAML configuration in the root of your project and then pre-commit install,
12:23
it will install all the pre-commit hooks for you. So now every time you try to commit your code, it will run all these tools. And if any of these complain, it will not let you commit. So sometimes if you just want to check
12:40
that whether these tools are passing or not, you can use pre-commit run. So I'm going to show you like how to run pre-commit. So, okay, so this is a sample project. So this is like, so this is our first example,
13:06
which we discussed. You can see that PyCharm is able to highlight most of the errors. So you can see that these errors are highlighted like unused imports and the inconsistent naming conventions
13:23
and like, so the variable which is not present. So this is the example after running, after running Pylint. So you can see that the Pylint is not like a strictly enforcing a standard for docstring.
13:44
And it is also not able to detect this error where we are trying to add integer and a string. But in the third example, I have added some docstrings according
14:00
to PyDoc style conventions and also type entered all my functions. So you can see that PyCharm is able to detect that. So let me try to run this. So, okay.
14:21
So you can see that I have this .pre-commit-config.yml file. So this file should be present at the root of your project so that pre-commit can configure these hooks for you. So you can see I have like configured Pylint, PyDoc style,
14:41
mypy, bandit and black. So you just, if you run, if you use pre-commit run command, it will run all those tools for you. So you can see it is detecting. So Pylint is failing.
15:01
It is detecting all those errors. You can see PyDoc style is also detecting some of the errors relating to PyDoc, relating to docstrings. Mypy is detecting that error relating to string and float. Black is complaining and we don't have any security issues.
15:25
So you can, you can fix it. And like, and then everything would be passed and you can commit your code. So lastly, if you ever have doubts
15:42
about how can you be like, you can write better code. You can see the Zen of Python by Tim Peters. You can spin up a Python console and write import this. So you will be presented with a list of messages or suggestions to write better Python code.
16:02
So thanks all for listening to this talk. And I wish you luck on your journey to be a better Python programmer. Okay. Thank you very much. So we have time for questions. Yeah. So first question is, sorry for the attendees.
16:23
If you click in the Q&A, you can write your question or if you want to ask the question live you can raise, click the raise hand button and I can enable the microphone. So first one is from Thomas saying, how you do deal with exception to rules where to draw the line, right?
16:44
Yeah. So like most of these libraries, most of these libraries comes with some kind of configuration. For example, like in Pylint, you can like use a file known as Pylint RC where you can like skip all, like all the rules
17:00
which are not in line with your organization or your team. So like every library has some kind of a configuration where you can like mention these exceptions. Okay. So any other questions?
17:21
We have time, so if anyone, okay. Cool. Then that's all. Thank you very much. Thank you for presenting. Yeah. See you next year. Yeah. See you.
17:40
Thank you.