Merken

Practical PyBuilder

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
but
these are not the enemy so smoking
yes no and so welcome to the practical part of the talk yeah and max obviously on my mind get top fight on his Twitter but I do have e-mail and so if you want to get in touch feel free to send me a lot of e-mails but I do speak German and French and a bit of English so you can use that to get in touch too I'm from your results 24 which is the leading journal real-estate portal so we can if you want to find a new apartment all moved to a new house we have services for that and while we're mostly a Java-based company so on most of the web development is done with a Tomcat in spring and so on and so really Java-based although we do have a few teams were to use Python fall development is of a cool guys from but where we think that you know pattern really hits list with spots and that system automation infrastructures coach so we do we yeah like everything in Python when it comes to automation so when actually do it is that I'm a system developer that we in the production department so you know where operations sets and so on and we we automate stuff with software development metalogic metallurgy so we test things and so on and the most high-profile tool we're working on at the moment is what you can see here and it's called jets which a series of reactions and basically it's a tool for system orchestration which is a lot like sensible on In many ways except that's Getchell the older and and of a difference so essentially is instable is orchestrated systems over stage based on rules like display book that you have to write and yet uses the fact that store or entire software platform was built on top of RPM and packages so everything is an RPM Package configurations not the software is an MP and and yet builds a dependency tree with all the packets and from the dependencies between was package is able to infer what needs to be done in order to update the server so the basically the idea is that unlike sensible where you know you execute applied to sort of uh yet child you have some hosts and you just tell updates and then
it does everything that needs to be done but in an automated fashion and over a lot of things so for error recovery to so it can heal itself and so on and it's pretty cool you should check it out so we start as a company and we're not so small company anymore so we have like 700 employees and about 150 maybe 290 so that's a lot of developers what you can see here is that the amounts of the 2 machines we have on 1 of our data centers and it's so like 600 so in fact we have like 2 thousands in total and what the slides basically tell you is that we're in environments where there's a lot of stuff going on so we have a lot of developers and lots of software and in that kind of environment so that the worst thing that can happen to software being a software developer is legacy i which comes in many forms and personally I think legacy means you can't touch it so you know maybe it has no test so you don't notice if you break or maybe it has no documentation and you don't know how to run the tests which is just as bad as having noticed at all and on basically 1 prevent that you don't want to see your software to become legacy and so what you do is you use Python ecosystem which is which really provides a lot of tools to to avoid so writing that software test frameworks dangerous like take this dates of these already cool tools that you can use to avoid to suffer from story wanting to its death and so so that's 1 thing to keep in mind you really want to enforce all these best practices and use as much of these tools as possible but the challenge basically is that we as a team decided that we want to empower colleagues from operations on we wanted to use Python instead of things like passion for because we think these things socks on so wanted to use Python instead because it's simpler it's more readable and it's easier to get productive of its a rich ecosystem and some of this the challenge we face essentially is that some of we do want these these these colleagues to use Python and we want them to use best practices so that for software does not become legacy also test that they should not right on the strips and so on and so forth and so the problem is you can show up and tell them yet you gonna use Python and this is a very simple it's a cool language and then I had this huge list of things that you need to know and understand in order to write Python code so you know you need to know how coverage works and you need to use that you need to know flake 8 works well you need to use violence and so on and it's this really overwhelming from beginners perspective so basically pipe is the solution to the problem on the beach what's which we we're actually aiming for as you check out a project from source control
and you TD into its and you have the 2 and with pipe installed and basically you just take 10 of which is short for pi and this service starts running so it doesn't things and that when it done and
everything but was importance has been taken care of and you know if you can check in or not so you run 1 comments and you know whether genocide would or not on so that's what we're aiming for here what you can see here that it's running integration test running unit tests are trying to actually flake 8 frost that's it's measuring the coverage during its efficiency so really it's it's fully automated arms how does it work well basically like every other bits will understand on its splits the into
tasks so you have these small building blocks of logic that have maybe dependencies on the Java and then you run high but and tell it what tasks it should as it executes and then it makes a plan with the dependencies and the required order and it does that on but that's still not enough actually because at that slide you still need the knowledge about the task that you want to execute so but still something you should have to
document and wiki or something and we don't want
that so the solution basically is of applied to the configuration file you have this defaults task thing here in the middle and some that's so what kind of should do when you just run it with tasks so when run private by default in this case is water and clean and then analyze and publish and that's what we use to make it just worked because we want people to just use 1 comment and then they know if you can check it on knowledge so um what I'm talking about so want to know is the mostly should sound like make probably aren't so that Max is a verse of those already make and make it is a much better than yes rights and I think the strength of lies in its ecosystem because basically everything Privatdozent plugin from so take for instance coverage and that using the conventional high API and the cool thing about these things being plug-ins is that's on you you don't just measure the coverage and you know it's not just you and the built and it tells you coverage is 50 % on that using the API so it can it can take decisions based on the coverage and use a developer can say for example 0 yeah I don't want the coverage to sink below 50 % and then automatically it's going to break the boats if someone like removes all the test because the coverage is going to be 0 % and that's something that's gives you the grantee as a developer that people are of following the best practices that you deems necessary to contribute to the project and so another good example is we have all of these Lynch's building on so we can we can use placates or potato in whatever and if you ever contributed to an open source projects in Python usually you make this contribution versus pull requests and then saw 1 of the guy with comments of rights comes up and say 0 yeah you know whether this is what the caterers and I'm going to make a comment on each of them and you're going to fix them well when someone contributes to 1 of my projects like idea for instance on my just a direct people to drive this talent look at Travis it's red because there's some that debate arose please fix them and when history we can merge and the cool thing is you can run the the full builds on this machine because Travis is also just running private without anything specific so 1 of the of the highlights I guess is that we use a value based configuration sold basically the idea is if you can understand English and read its then you should understand what each of these lines stores and for instance the 1st line of configuration here satisfiability around the integration testing parallel and you get you have 2 lines of like here and that OK to ignore some specific frosted warnings and also like you probably noticed this is Python also we think it's pretty cool and a lot of people have criticized sunset apply for being a Python file because people actually abuse its and pretty that weighs on personally I believe that was part of this is not a problem on because we we don't have the same problems are set up I I I don't have time to get into details but essentially it has to do with the fact that that set of pi is packaged when you do a source distribution and the pedophile is really just for development on and when public packet something it actually writes a set of 5 5 so you can be pretty sure that the fight is going to be extremely keen on another thing is that we have the requirements built in 1 so in your had a conflict can be spilled up I file you can actually write so what you're project depends on in a declarative way so you can also super rates between between normal runtime dependencies and the dependencies so that's pretty cool and 1 of the 1 of the nice things I think is because Bible is using paper neighborhoods right you can you can put specific that configuration inside a project file and for instance like you have the last line you can tell pit yes vote for this project are not going to use the public by the server but instead instead of you gonna use or a local installation of the death defocus is over I measured on the DSP by way from public regulation check it out so that 2 points you just can't so you can do all this of this instrumentation stuff on on top of it and this is really something you can do with requirements that txt because requirements that takes 2 years just new line separated file with requirements and you can tell it to use another index just of more so and this is the last slide I still have a bit of time so I'm going to switch to a shallow and show you how to get started on the project from maybe used this well so and so some questions so and then to a temporary directory and actually I don't trust the Wi-Fi
you i'm because dropped a few times already so I have vigilance at which is also already loaded with everything and to to show you so the most important thing is that the hydrogens installed on the
something you should always always put individual and because these to see the package and the vitamins and I'm going to make projects and name ATP 14 on so basically an argument end of it's fine to fail because you know it's not project gets so we have this nifty little tools which is called start projects that will start start of course on and this often if you questions on so mostly the ideas you can just press Enter all the time and because it's going to suggest defaults and so we have like this defaults project structure which tells that's of resources during 2 swordsmen Python from from a unit test going to source unit has Python of strips going to source scripts and so on and just going to stick with the default here on and and the the scripts that this I think pretty important to note subscripts are separated from the source codes on because I had
the Brewers was talking yesterday about building or some command-line applications and where of the referee basically said yes scripts or like grandparents because live forever and personal observation and most cases fails to look like shit so we don't want them to be in the sources and you know really keep scripts to a minimum from so that's the idea and projects and I can run its and its failing because basically 0 % coverage I also have 0 persons sources and so am I going to say that and again at key value based configurations so really it's that's the that's the easiest thing for us said property and this property is called coverage rate builds 1st documentation for that and I just set it to false and candles and it's going pass so it's alerts me about the privileges that actually really low but it doesn't break because that's a choice I can make as a developer on so know what I'm gonna do is I'm going to create a script general scripts that this is the easiest thing to do right now on so strips and I'm going to college EP 14 and start with Python she
users of it and I think you should also also all the time you such then unless you specifically once 3 or 2 on because it's this works of to uh and just kind Princess I'm so I'm going to run through that again and then what happens is it that builds a source distribution and this target folder and what what I can do is as simply install it target so so it gives you didn't know you can can install target sets of which would who have no looking looking I have this 14 scripts which has been installed in my vitamins and just like that so unlike was set up by didn't have to tell and I didn't have to tell the 2 would that's this is the strips and just drop it in the right place and it just works so American components solids and so next to next thing I'm going to do is moving a bit of a strict into a source file and write a test for its a sole-source main Python yeah that's the start of unit test From the 14 test sample from you this so this is what Nate note that you can also use pi test on those tests or whatever so a few other use asserts with Pakistan works to come and just come write simple test on that should asserts that yeah I don't know so such equal i'm gonna call it's cannot be 14 but like Europe and I expect this to to you string which will be your Python so as to the important so now I have a test which is obviously going to fail because they don't have the 14 pick it yet so I can can check it out and it works because I need to test for yes and so convention for a test file names to and it's a glob and the default is if you 14 is the star so what current underscore tests with an on so I have to name my files like this on this is also something you can change but again I'm just sticking with the default and yes no so I run the tests and it says you kind import 14 so I'm going to create and drop in enforcement Python so not scripts are and it's going to put everything in path and basically it's just that this function you apply and that's going to
return something else right now so return not because I want the test to be so I can test and you can see I I didn't have to tell piepowder but what's what where my tests are just run its and it just works because most of the tests are on notice is reds and have to fix it and so now I can return like your patterns and know it's still works on yes so basically the problem I have here is that I called my package exactly on like the projects so essentially what you see here is that on the package conflicts with our of actual source distribution so I have to fix that to i'm just gonna currently probably so as to rename this test to right and of course I have a 100 per cent coverage which is not very difficult on and yes that that works so but must done with the demonstration I think I'll take questions no maybe of questions which are related to whether adjusted so feel free to ask away and the question
was if we have things integration but not right now no and it's probably just a matter of writing a plug-in because of the fingers things largely automated so we don't have to do a lot of things but but that's was no not quite now so yes just this things some so when a company we used in city which is basically Jenkins with that CSS and its cost shit load of money on but some wikileaks that use Jenkins with its we also use Travis wouldn't essentially and the cool thing is that some when you decide to use a CI server itself if most of the time it's difficult because you have to you know you have to find out what you your project dependencies are for you talk to get them on real CI server and so on and then with Pagliari you just 1 vocabulary commands and it's exactly the same as on the workstation the 1 and yes so so the question was how the reporting works on basically you have 2 choices vary from 1 of the things you can do with is 1 of the most CI service you can see the output of a comment on so you you have the output of private which tells you what's wrong and the other thing you can do with is and you have this this target directory and inside the inside the sphere directory versus reports directory and there you can see for example and jason formats or you know text of example code if you interested in coverage and you can see it like you know this 1 2 statements and 100 100 % coverage numbers also the same thing with J is on so you can reuse that's I yes so on most CI so there's also actually and north to the city but I I think Jenkins can do to you can like you can upload artifacts after bills on and so what we have in cities here is just put up all these reports on the on the boats on so that you know if it fails you can look at for example the coverage or the flat case and you see what's wrong does that answer questions any more questions b b
Code
Computeranimation
Quelle <Physik>
Resultante
Bit
Momentenproblem
Extrempunkt
Datensichtgerät
Formale Sprache
Versionsverwaltung
Schreiben <Datenverarbeitung>
Aggregatzustand
Extrempunkt
Netzwerktopologie
Rechenzentrum
Metropolitan area network
Mustersprache
E-Mail
Chi-Quadrat-Verteilung
Softwaretest
Nichtlinearer Operator
Reihe
Ausnahmebehandlung
Biprodukt
Rechenschieber
Dienst <Informatik>
Twitter <Softwareplattform>
Menge
Server
Projektive Ebene
Reelle Zahl
Ordnung <Mathematik>
Programmierumgebung
Maschinenschreiben
Subtraktion
Total <Mathematik>
Systemplattform
Code
Framework <Informatik>
Virtuelle Maschine
Bildschirmmaske
Software
Perspektive
Speicher <Informatik>
Softwareentwickler
Ganze Funktion
Gammafunktion
Normalvektor
Fehlererkennungscode
Metalogik
Schlussregel
Mailing-Liste
Physikalisches System
Quick-Sort
Mereologie
Web-Designer
Personal Area Network
Softwaretest
Metropolitan area network
Bit
Dienst <Informatik>
Uniforme Struktur
Komponententest
Computeranimation
Integral
Gebäude <Mathematik>
Applet
Besprechung/Interview
Automatische Handlungsplanung
p-Block
Wiki
Mathematische Logik
Task
Rechenschieber
Metropolitan area network
Mailing-Liste
Task
Vorlesung/Konferenz
Ordnung <Mathematik>
p-Block
Mittelwert
Nachbarschaft <Mathematik>
Distributionstheorie
Abstimmung <Frequenz>
Bit
Punkt
Versionsverwaltung
Computeranimation
Task
Metropolitan area network
Virtuelle Maschine
Task
RFID
Vererbungshierarchie
Biprodukt
Softwareentwickler
Speicher <Informatik>
Automorphismus
Default
Parallele Schnittstelle
Konfigurationsraum
Stochastische Abhängigkeit
Regulator <Mathematik>
Gerade
Inklusion <Mathematik>
Softwaretest
Open Source
Gebäude <Mathematik>
Rechenzeit
Plug in
Quellcode
Elektronische Publikation
Bitrate
Menge
Entscheidungstheorie
Integral
Rechenschieber
Diskrete-Elemente-Methode
Menge
Automatische Indexierung
Rechter Winkel
Mereologie
Server
Projektive Ebene
Normalvektor
Verzeichnisdienst
Lie-Gruppe
Instantiierung
Offene Menge
Komponententest
Extrempunkt
Fächer <Mathematik>
Kartesische Koordinaten
Oval
Extrempunkt
Information
Computeranimation
Intel
Metropolitan area network
Verzeichnisdienst
Einheit <Mathematik>
Eigentliche Abbildung
Kontrollstruktur
Skript <Programm>
Luenberger-Beobachter
Booten
Datenstruktur
Softwareentwickler
Default
Konfigurationsraum
Auswahlaxiom
Gammafunktion
Inklusion <Mathematik>
Binärdaten
Parametersystem
Pay-TV
Kategorie <Mathematik>
Gebäude <Mathematik>
Quellcode
Bitrate
Kreisbogen
Vorhersagbarkeit
Projektive Ebene
Versionsverwaltung
Message-Passing
Offene Menge
Stereometrie
Distributionstheorie
Bit
Dualitätstheorie
Komponententest
Euler-Winkel
Versionsverwaltung
Oval
Extrempunkt
Computeranimation
Metropolitan area network
Trigonometrische Funktion
Verweildauer
Total <Mathematik>
Stichprobenumfang
Mustersprache
Skript <Programm>
Installation <Informatik>
Zusammenhängender Graph
Gravitationsgesetz
Default
Gammafunktion
Softwaretest
Binärdaten
Inklusion <Mathematik>
Lineares Funktional
Gruppe <Mathematik>
Logarithmus
Quellcode
Elektronische Publikation
Wertevorrat
Kreisbogen
Differenzkern
Datenerfassung
Projektive Ebene
Versionsverwaltung
Einfügungsdämpfung
Befehl <Informatik>
Zahlenbereich
Code
Computeranimation
Integral
Dienst <Informatik>
Kugel
Last
Reelle Zahl
Arbeitsplatzcomputer
Server
Dateiformat
Vorlesung/Konferenz
Projektive Ebene
Verzeichnisdienst
Verkehrsinformation
Auswahlaxiom
Funktion <Mathematik>

Metadaten

Formale Metadaten

Titel Practical PyBuilder
Serientitel EuroPython 2014
Teil 15
Anzahl der Teile 120
Autor Riehl, Maximilien
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/20007
Herausgeber EuroPython
Erscheinungsjahr 2014
Sprache Englisch
Produktionsort Berlin

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Maximilien Riehl - Practical PyBuilder PyBuilder is a software build tool written in pure python which mainly targets pure python applications. It provides glue between existing build frameworks, thus empowering you to focus on the big picture of the build process. It will be shown through demonstrations and samples how a simple, human-readable and declarative configuration can lead to an astonishingly well-integrated build process which will make maintainers, developers and newcomers happy. ----- # Why another build tool Starting up a simple python project with best practices still takes a lot of boilerplate and glueing (e.G. chaining unit tests and integration tests in the build process, adding a linter, measuring coverage, ...). It often results in extremely ugly homebrew scripts and edge-case solutions that are not reusable. There are even programs out there (e.G. cookiecutter) that encourage boilerplate code generation! # Build orchestration PyBuilder borrows from the *maven* idea of phases (packaging, verifying, publishing, ...) to set up a fully declarative and automated build that can be run locally and remotely (build servers) in the very same way. Rather than reinventing the wheel, it provides glue between existing solutions (like unittest, coverage, flake8, ...) through a simple but powerful plugin mechanism. # The talk After a more theoretical talk with a colleague at PyConDE 2013, I want to show how it's actually like to work with *PyBuilder*. This includes * starting up a project * running builds * using plugins * writing a plugin
Schlagwörter EuroPython Conference
EP 2014
EuroPython 2014

Ähnliche Filme

Loading...