Automate cleaning code in few easy steps!
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Anzahl der Teile | 112 | |
| Autor | ||
| Lizenz | CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 4.0 International: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben. | |
| Identifikatoren | 10.5446/60770 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
|
EuroPython 20226 / 112
5
10
14
17
19
23
25
29
31
32
34
44
47
51
53
54
57
61
69
70
82
83
88
93
94
97
101
105
106
00:00
SpeicherabzugSpeicherabzugSoftwareentwicklerSelbst organisierendes SystemMereologieDatenverwaltung
00:34
SoftwareentwicklerVorlesung/Konferenz
00:51
CodeParallele SchnittstelleMenütechnikKonfigurationsraumVersionsverwaltungVariableWeb-SeiteComputersicherheitSoftwareentwicklerSelbst organisierendes SystemCodeBitMathematikDateiformatVersionsverwaltungDatenverwaltungSoftwaretestHook <Programmierung>MultiplikationsoperatorElektronische PublikationRepository <Informatik>GeradeProzess <Informatik>ProgrammfehlerTypentheorieMailing-ListeProgrammierumgebungGrenzschichtablösungKonfigurationsraumComputeranimation
09:17
RechenwerkE-MailMAPInstallation <Informatik>Physikalisches SystemGarbentheoriePauli-PrinzipDateiformatDatenverwaltungStochastische AbhängigkeitWort <Informatik>Witt-AlgebraPlug inGruppenoperationFunktion <Mathematik>Repository <Informatik>DatenverwaltungKonfiguration <Informatik>KonfigurationsraumProgrammierungProjektive EbeneElektronische PublikationWurzel <Mathematik>UmwandlungsenthalpieSoftwaretestVerzeichnisdienstProdukt <Mathematik>CodeDefaultGruppenoperationSkriptspracheVersionsverwaltungPhysikalisches SystemPauli-PrinzipBitMAPJust-in-Time-CompilerInformationFormale SpracheTemplateVirtualisierungComputeranimation
17:43
Auflösung <Mathematik>Demo <Programm>SondierungProgrammbibliothekProgrammierumgebungSkriptspracheProgramm/QuellcodeXML
18:35
SondierungSoftwareentwicklerVirtualisierungComputeranimation
19:35
Sigma-AlgebraSondierungSoftwareentwicklerTwitter <Softwareplattform>Elektronischer DatenaustauschComputeranimation
Transkript: Englisch(automatisch erzeugt)
00:06
I'm Esther, I work for May.com, a UK company that designs and sells home furniture online and in other parts of Europe as well. And I'm also a Pico and Italia organizer and developer.
00:24
I recently joined to start Bari GraphQL as a core dev and also as a Poetry Packer manager contributor. So with this organization, I have seen a lot of tools and automation that I think they
00:42
are really great and help to have a better developer experience. So today I would like to show them to you. And oh, we are sale, by the way. And so you can go and check if you like something. I should also mention that my team has recently fixed a little thing that about Ireland post
01:11
validation, so now we can ship also to Ireland and other countries. So go check online, please. So have you ever been in this situation when so you're working on a feature, you
01:27
are writing some code and I think you finish your work, the tests are passing locally, so you are ready to commit your code and push your code, and at this point, I think
01:45
we have some time to kill because the big blanket took a while, so we can go out for a walk, relaxing, get distracted. At some point, we will remember that we have to work, and we will go back on the pipeline
02:04
and see that everything is red. We forgot to run the linters locally, so the pipeline warned us that the code quality is not reached, and we should have learned our lesson, and next time, we will remember
02:23
to run the linters, but the truth is that we probably forgot anyway at some point, also because sometimes you do a silly change that doesn't, shouldn't change so much, but
02:40
that silly change that breaks black or I sort, so we probably waste a bit of time because the pipeline took a while because we never optimised Docker, so on, anyway, we can optimise a bit the stuff and introduce our first tool, that is pre-commit, pre-commit
03:06
is a multilingual package manager for pre-commit hooks and it will run automatically on every commit, before every commit, so we have just to create a little bit of configuration and there is a nice command, sample-config that we create for you, and we have to run
03:25
pre-commit install, so this is our configuration file, and you can see there is the repo, there is the version, because pre-commit will install the stuff in a separate environment
03:43
and everyone in your team will have the same version, and there are the ID of the hooks, so there are many things you can add to your pre-configuration, and I think in a Python
04:02
weapon, you should always add at least black, I sort, and flake 8, but there are many, and the list is very long, and you can check online. Those are some of my favourite, so there is py upgrade that automatically upgrade the
04:23
syntax to a newer version of Python, so our code is up to date and hopefully will be a bit faster, and there is flake eradicate, so that will notify us, we will have some
04:40
commented code, and the last one is a nice one, because flake 8 or py lint only tell you that there is a missing, there is an import or use of variables, but they didn't remove it for you.
05:01
Instead, out of flake, we removed it automatically. So what else we can add? The list is long, but it's not always about code quality, it's also for our safety, because there are some dangerous stuff, some secret, oops, sorry, this is the current one.
05:24
So it's also for our safety, because we are very careful to not commit some secrets and dangerous stuff, but the truth is that when it's committed, it's kind of too late, because you will leave it there, kind of forever in your history somewhere.
05:42
So this is just an example of what type of stuff you can add to your checks. Why we should add pre-commit? So I'm trying to answer this question, and one of the benefits is because it will help
06:04
to identify the silly issues, like leaving some of the bugs in the code, or missing a new line at the end of the code, also because this stuff only appears during the code review, because GitHub did a very good job to emphasise that you're missing a new
06:24
line at the end of the code, this tiny icon, and so in my organisation, there are I think a lot of these comments, and if you use pre-commit, you will remove earlier,
06:41
so we can concentrate only on what matters, so the code, what has really changed. Also to me, it's a bit awkward sometimes when I say there's a little mistake, because I wonder myself, should I bother to notify the author, or I should look in another way,
07:04
because the work will, the code will work anyway, so maybe I already comment a lot of stuff in the pull request, it's big, and yes, so we can avoid this stuff.
07:22
Also, it's because nowadays, we have a lot of code, it is not only Python, we have no matter if you have a mono repo or microservices, we have Python, we have Terraform, infrastructure, or Ansible, some bash scripts, some markdown, because we have documentation, JSON file
07:47
for fixer, anything, so it's multi-language, so you can add a hook to format any code that is going to be read.
08:02
So I tried to ask myself why people don't like it, because of course I'm a bit biased because I love this tool, so it's difficult for me to identify why, and I wrote a tweet, and maybe my bubble already uses it, anyway, I think one of the main reasons is because
08:25
people think it is slow, but I don't think it's the case, because of course the first time you add the precomi to your repo, and you promote all your code base, of course that can take a while, but normally it will take only a few seconds, especially because
08:49
it's incremental, so you precomi will run only on the code that you are about to commit, and also because, for example, if you commit some Terraform code, it will run
09:03
only the hook for Terraform, not for Python, so it's very quick. Anyway, it can be that some hooks are slower than others, for example, mypy, that hooks
09:20
that has to analyse the code can be slower, of course, and you can skip it, or you can run one at a time, you can skip there entirely, and it's okay anyway. For example, if you're doing pair programming and you commit frequently, you don't want to bother to the code quality, so it's okay.
09:42
You can run later if you want. So there are options to mitigate if there is this problem. Also, as you mentioned, they add a manual stage, so by default, precomi will run on
10:02
every stage. There is not only commit, there is precomi, post, commit, premerge, postmerge, there are a lot of them, and they introduce a stage, so if you have something that you have to run only once in a while, you can run it manually, and this is a nice feature, and
10:28
yeah, the only thing now that you have to remember is to run precomi install when you clone the repo. The only thing. We can potentially avoid if we do a nice trick, so if you add a template in your home
10:49
directory, in your JIT home directory, so when you clone or initialise a repo, there are some files that will be copied from this default directory of JIT into your .JIT
11:05
directory, so you can add a little script in your default directory and it will be copied, and it will install precomi for you, so after, you don't have to remember anything, and
11:21
if the repo you clone doesn't have precomi, doesn't use precomi, you will just print something and leave it there, so cool. So I think one specification is needed because using precomi or not using precomi is your
11:41
personal choice, so you should not force your team to use it if they don't want it, and I think at the end of the day, whatever makes you productive, it's okay, so choose what you do, every commit, every push, there are many options, whatever fits for
12:01
you, it's okay. So, what else we can do? We can run of course on our pipeline, because we want to double check the code quality, of course, and there is also precomi CI, and it's nice because precomi CI can also
12:24
edit the code for you, so you can also fix the hooks, so check it out, and I think for public repo it's free. What else? So if you're like me in your organisation, have a makefile in your project, so every
12:47
team can decide what to put inside the makefile, so we ended up with a bunch of commands that they do always basically the same thing anyway, so at this point, if you
13:01
have precomi, I don't think we use them any more, so we can clean up our long makefile and, yes, clean it up. So we have one tool to run that all, and that means that we have a lot of hooks now,
13:22
so every hooks need a little bit of configuration, and this configuration is stored in a file and usually in your root directory, so that means we have a bunch of dot file dot CSV in your root, so we can think about cleaning up also the repo and your root, and move
13:44
it in a special file called pyproject.tom, so what is pyproject.tom? So at the beginning of Python, there was only one tool to build packages, but after
14:03
some point, set up tools came out and created some issues, so this PEP 518 managed to solve these issues, and after, every project can specify which build system they
14:20
want to use, there are many, but it was a matter for us, it's because there is also special session for tools, so we can add there the configurations. Not every tool allowed you to use pyproject, many of them are already integrated, this
14:48
is not developing or migrating to this, but you should check their documentation if they support or not. I talk about specifically about pyproject.tom because I want to introduce another build
15:04
system that I think is amazing, and it's called Poetry, it's an all-in-one tool to manage dependency, virtual environment, packaging, versions, so some languages like Go or Rust
15:23
has a tool that manages all the aspects of your project, but Python doesn't have a tool like this built in. So Poetry managed to compensate this, and what does it mean for you? It means that you can delete the setup UI, that is deprecated by the way, requirements,
15:46
setup CFG, and other files that you have, and have only one pyproject.file, so how it looks like this, hope you see it, so you can see at the beginning there is some
16:04
information about metadata, the description, the versioning, we have the dependency, the specification of the dependency, and they are grouped by the production dependency above and below the dev dependency.
16:21
These are only the primary dependencies, so those you add to your project, because when you install, when you do Poetry.install, it will create another file called Poetry.lock in there, you have this exact specific version of your dependency, so in your team,
16:42
when someone installs the project, it will have the same versions that you have. So this is easy to debug the stuff. Okay, I should mention that Poetry 1.2 is coming very soon, hopefully, and will introduce
17:06
some nice features like group dependency, so until now you have only the production dev dependency and extra dependency, after that, you can decide how to group and name
17:21
your dependency, so you can create the test dependency, the docs dependency, deploy dependency, whatever. It also has a plugin system, so it means you can customise this tool on your needs, so it's very interesting, and very looking forward to this.
17:43
And, yes, this is how it looks like, Poetry install, it's very fast because it's paralysed, and there are many commands, so Poetry add with add dependency, Poetry update, Poetry
18:01
run your, for example, if you do Poetry run Python, your script will run the script in your environment without the need to activate your environment, and this is kind of cool, because you don't have to activate and wait, it can be slow, I'm very slow when I have
18:21
to activate the environment, so this is really great. And of course you can publish the package if you have a library on PyPy, just with a simple command. So, yes, the very last thing I have to say is that JetBrains has done a survey recently,
18:46
I hope you can see it, it's wide enough, but, yes, there are many questions in it regarding the development, and it's very interesting, I was very surprised to see how much people
19:00
are using Poetry already for dependency, for packaging, for a lot of stuff, for virtual environment, so it's kind of, I think the third tools depends on the need, depends on what you are looking for, but, yes, it's pretty great, I think.
19:27
Okay, I don't know what's happening. No, I finished my talk, I don't know why it's not, oh, okay, okay, leave it there.
19:43
Thank you so much for your attention.