Merken

Packaging and testing with devpi and tox

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
OK so some like
high-dimensional already have been doing a number of tools and and testing test and then after some time I decided OK and this whole thing about unit test Pieter most and what not and actually would be nice to have really unifying experience when running tests against the Python application that's why I went for also writing talks which is kind of like a mutant has and that can actually invoker most of the mutants of applied and after a while I thought yeah that's all very nice but the real power problems when you want to have something like quality assurance in your projects it's really also bought release management so you actually have several packages dependencies and I have that with my own open-source
projects but also with people I think companies like consult for and that's why also went for a given next level to have something that manages the packages and also but also the test but all the time coming very much from this kind of like USA and testing perspective so that's where you get by
actually I was born the the systems
basically there to help you with pipeline related to reuse workflows and quality assurance it currently consists in a
conversion to 0 of 3 main components which is the core of that piece of and we're going to talk about all of these components in detail the sum of the lecture provides the pipe eye-catching index and your private nexus where you might not actually want to publish from that if you want to use that within your organization it has the recently released as the death by web plug-ins which um provides a web um interfaces also for your documentation a few other things and search across MIT adult and documentation and then there's the 3rd that you don't have to use actually but it's uh and helpful if you have to deal with the development and production indexes and so on and that's a command-line tool that basically drives the well known in other and tools like paper pen is install and and set up the pie upload and things like that so death by indexes 1 of the main purposes at the beginning that was before all PPI people from all new content delivery network was that you can have a local themselves updating pikey cash so you basically work against your local indexes the packages not there because of 2 pi pi grants and the next time you don't even need to be online again you don't need you need to have online connectivity it'll just satisfied everything completely offline from the local so everything that you install basically gets cached during the index information and it uses the change like protocol piety so that it from time to time at pipeline is there anything new for the projects I care for if so it basically invalidates the cash so the next time you as it's going to update the cash as with every cash cash litigation is a very important topic and this is actually using the official had 3 8 at 1 API it also manages multiple private clinics for you you want to implement staging and each of these annexes um supports running against that with people you install although culture and it supports the typical of set up by upload upload box and so on commands call you
can then get packages into and that staging and there's
1 feature that distinguishes and FPI from other things other indexes that you may know in that it provides an aggregation or inheritance future so here
this is 1 possible layout that some people use and you have the so called Group that's the cash I talked about we can directly use that if you don't have a private and exists and forget about the rest but here we actually have a production index which contains the private indexes of the private packages that you um want to publish and piety powerful which bot which might depend on PI PI and release
550 don't have your private and next so you may have and application that depends on parameters and improvement depends on that of other things and those all come from root but if you work against the company production index is going to see 1 unified view of your private packages and all of the PI PI PI packages and then if you want to do some kind of to a from you I'm also can do a development index for example team based with some companies are doing and they just put during development releases that are not ready to be deployed on web service maybe but they can be used for further testing and on 1 important thing here is that so your production index is actually somewhat protected from malicious PI PI packages and going to tell this which is also interesting if you don't use the FBI something which I call the higher versions of tax is also variants of this attack let's say you have a credit cards and released files that contains your credit card processing In Europe education to put this on a private index and somebody that the attack actually of nodes credit cards with a slightly higher version number 2 pi k I now if you say if I install against the the against the production index that inherits from group IPI with this and command I'm actually going to get the PI PI because I didn't know that somebody actually went and occupied my private name and piety of PI PI is a package which anybody can basically publish any kind of package so here of private partitionings but are not yet registered at party I somebody can go there and other series and the the I don't know I didn't find myself but I'm pretty sure I could get I get something like 100 watts per day also
with something like this but not the only problem that is there but I'm just saying that if you have something that some home merges the world of the pie Python or can be with your private and next then you you get into this kind of problem and it's also the case actually if you forget about 7 and also the problem is used that install extra next year because then the margin is actually done on the client side
but it does exactly that it actually takes the higher margin so you and that you thought you install something from your private index but you actually installing something from from PI PI so that's a bit of a problem that the iron version 2 that's role prevents that because it says by default if you upload anything to a death PI private and X any kind of further look up you if you inherit from the by the I catch will be prohibited and you have to white list if you actually have a package
that comes from public private walk across the open source for use of your company then you have to whitelisted otherwise all quite guys ignored risk for acute basically install from the production index credit cards and it's not whitelisted so by default PI PI is not considered because there is the package in your private index so it's basically trying to prevent this kind of error all but but that's not the only way if you want to be a bit more
careful because there's a lot of text and for example if you have if you have typos
somebody in your company and the laptops installing promote without a d the end of what I do sometimes pipette install pipe test so if you want to get hold of my machine it's very easy because you just need to register the package pipe test property for some reason I sometimes forget this last letter foot currently registered so the chance the Commission of so if you actually want to this is really a
problem because I mean you can imagine there's some very popular packages if you if you register of variance of this kind of package names you will eventually from the millions of users literally across the board you will get some people actually I check with the pikey admins and I actually you can see that the sum logs and they are actually a lot of instances of this type of thing so it's clear connects the point that a but this is not about attack those against by the would be a fun talk by itself and this is about if you want to be more careful then you probably should not inherit directly but you rather have would pipe the set that cash and you work with that and development but then when you want to have a package in your interim dependencies in your company that push it explicitly into your production index and that's a renewed development index right and then basically you just push packages around the and that's something that FDI makes easy somewhat easier and you upload your own packages the company death and you won't have any kind of these detect problems um like typos and so on and suddenly people you your production machines cannot be easily compromised this is just some background how you can organize
images that might want to be careful about regarding the the the the way how you can
organize the nexus for your teams and also may be platform-specific indexes that contains the yields for you before deployment platforms and so on there's several variants about this and kind of best practices and lodging which document but this is kind of a start on this so 1 feature that came out uh came out last week actually is replication because that's what to 1 funding company which get some money for development for the open land wanted to have is that you can now run and FP 7 replication note that means that the 1st command accused of the full command that you run on part 3 thousand and then use that replica somewhere else you come in this case just also mobile host specify that the service data goes into a separate directory replica 1 and then I say OK my master actually is this so the 2nd location actually starts of replication instance so and this works by entropy between the um really kind master and it retains full failover copy so that when you actually upload something to the master it's you can also upload something to the rich has the full interface and that will only compete in packages also at the master so at any point in time where you upload something you will have at least 1 to host and in all right so it's kind of like a simplified replication model I'm always go through the master and that kind of seems to work quite well already although there might be some but it's called Latinized Greek and Latin running it to myself inferences and some companies are starting to use the replication also in this setting a and that by weapons this the 2nd big feature that came out last week from mostly implemented from Florian received there the edge of the fact that the FBI to use parameter everywhere and that by web actually is very nice that interface now that shows you will meet at that time and and summary information description and documentation so it's basically read the docks and read the docks In the company basically so as well and um shows that
quickly so you so this is my senior public for instance
this is like for example my development index and 1
of the things you see that for example the 201 release with that that's the reuse file and you see some
tests that were performed on the various parts of the true platforms hearing 42 and knocks on the different temperatures and I can basically look into that and see that
I this was executed and um In the same way of
course would see there's a failure somewhere also if I
have um documentation I
can go in here y can just say show me the what we know about death by and Jenkins not the full index the full diff piece search
and then I see ok there's some some links to that and
I get to the integration part
and with Jenkins on the death by documentation and that was that is just there because I uploaded the documentation for the index it gets unpacked you get the wells for that and it's indexed and the in the search so
that's also quite time and
powerful facilities so the last component is staffed by client and it's a relatively thin wrapper around the pit and some set up by the implications but also performs the actual upload so it always uses SSL and some other bits and it maintains on your local machine any kind of look information so if you say OK look in and then I use a certain index and upload something and then I don't need to looking all the time because that token I get from the cell that's going to be valid for 10 hours and have declined basically stores the temporary authentication information it also has experimental support no for l certifications if you want to step pure and scenario um to have uh encryption and authentication through as is the commands that that's the kind of us are used to actually set the index you and what I'm development or just group IDI on production server other is for helping you with the uploading files and docks and so on from check out test is the 1 that produces the tested evokes talks actually and pushes the operation that actually pushes every use including all of its documentation and release files from 1 index to the other and paper or other installers and disparity use and then there's some configuration administration commands that you can use for index configuration user configuration and I'm also accessing the adjacent interface so that server as a full Drazen interface and all of the resources that you can use for scripting um that typical release workflow is
looks like this you basically go to your development index you upload release file you implicitly built because you in the set up the pie directory you just implicitly built with that pair upload or you already have built during his father and just say that upload this release 5 and you send it off to the index and then from the same machine all from all kinds of other machines that you might manage the Jenkins or something you issue this single line death pipe test package names and that actually gets the latest release and performs the tests and attaches the test results back to the reduced file that's why I could see in this review you OK this release file what kind of test has it seems that was produced by this the giants side if I just lunch and when it's ready actually when you're happy then you push it to another index and of course you can also automate this kind of like the Jenkins job and just these commands To on success of something posted to an index that says this these are all the test the test passing packages and things like this um so this is a really that gets slide shortly into talk um stocks is a tool that allows to define how you want to know what kind of tests you want to do against your release file it's basically the release file it expects to find the top but I and II and then it invokes talks I over the next slide discusses what that means produces something called talks result adjacent and of them I can actually from the command line I can say that pi list the package name and see what the status of it was the best possible what kind of test failures that 1 show me the trace back from the command line and then I take the the release file once I'm happy with that this is then put pushed bit by bit of other term to the next the next 1 will that this thing I actually tested against on the different platforms I'm actually works like put this thing I don't basically re-upload something to production I really take the same thing that works and push it through to the next stage In talk
faltering test runs um it's
kind of a centralized testing and not going to talk much about this because my slot was exchanged for 30 minutes and talk it was originally a 45 minute talks most of it from you so I can talk too much about it here um but you can go to the and web page to actually get some more information about how you configure your test runs with different from a and so you already saw that to basically just installed a server you and have a typical host plot and some other settings that you can and the date idea where you want to have some and then from from different clients that don't need to install the piece of of course you can't just to install 50 giant and then say that I my companies over and just let against that but i which is usually want to do is that you want to have an engine X space deployment there's a
an example file that gets generated from your settings host port and so on and so on which is basically a problem on his complete and genetics of basic engine genetics side config file that you can just you'd in Europe and annex configuration used as a template to work for the from and it's this actually happens in such a way that engine X directly the static files so
something is actually that the seller doesn't see any more once you upload something the whole euro structures such that the and next directly such that so and for that and that the seller doesn't need to be running so I'm going
to conclude the death systems and
developed since about bit more than a year I think a year and a a couple of months it's MIT licensed it's test-driven development about surprisingly and and also it's a bit funding driven so there's some users cases that are interesting to me myself personally but it also depends on whether the upcoming things maybe as a company who found some held up integration of indication integration but kind of like future development and some things and consulting is provided by and of course for the tests are good way to contribute the debate that's my brief overview what of planned talks thank you we have a good
good 5 minutes of questions you just briefly talked about edible syndication and doesn't mean that you can integrate this tie into an Active Directory domain and use this information to authenticate users well if the funding realizes against so as the Knight of trying to you if you can give you some money the source of all of the the yes I mean this brand or something like this is also possible but you must print and you know take some time organization and get a lot of get something really is ready and documented and everything and you probably know that that's kind of some work involved right and but just to give you a brief some idea on how the the future discussion around 0 that is currently such that we say we want to have some 121 to have and genetics deal with a lot our integration and just past a certain user name header and 2 group header into that the servant basically have an option to be some that just says OK my upstream and index is going to pass me the right thing and engine explore the integration because sliced lichens for in the next but actually do this and then we need some client-side support to handle the look in part but that's that's kind of like the current implementation the other the account of obviously is to actually have direct lot support and FBI summer itself but well you don't have to reinvent every real against this high thanks for for all this hard working them and the question is about to come this is my decision run by the Forest Service In particular I am is it possible to configure some workers sharing more to it so itself because of data and a world quoted from produced over yes I mean maybe it wasn't clear enough the um the survival and the running of the tests for example they are completely separated so when you issued death by test this completely separate from the of around it's the death by just the money goes to the Sullivan gets the files performs the testing on whatever holes and then attach respect the test results so on the digitized server instance itself with the summer runs there's nothing there's no there's no set up but play or anything and executing otherwise it would be bought by mean you have to execute something like set up the pie you basically run the risk of compromise who has you know the pushing is really after you test that you tests like what you saw and that's from the apple will
be uploaded also do on the client machine I mean the client machine that's
the building and the like you reveal for example for Linux of 1 to 14 for a 64 bit blah blah reduces to come what and then you actually have motor resulting file to maybe a platform-specific real index no it doesn't know it doesn't although there is an apple so you upload you can define and apply index spaces and it and talk about all the features you can power index actually if you upload something it can for example trigger Jenkins job it's kind of like 1 path that this document it I show it to you just go to the documentation and then the succession section about the tendency of integration some of you already know the answer my question I have already have some of our dependent system system all signaling stuff like this Jenkins plugin from is it already as generic as I can and may be generated in files from this uploads trigger I mean death by tries to solve a few problems but only those it's not yet something like it doesn't have like all kinds of events that has this public trigger for Jenkins but not a generic the book whatever so mean that the very hard to do but it's basically a very much the FBI is very much driven by actual real world use cases but by all the features I can possibly think of so also when somebody actually comes along and wants to have a certain feature and know discusses the use case it's it's much more likely that it gets implemented that's kind of like my general development approach these days OK 1 mobile all good and that's it thank you very much
Softwaretest
Signifikanztest
Komponententest
Open Source
Zahlenbereich
Kartesische Koordinaten
Signifikanztest
Computeranimation
Berline
Datenmanagement
Reelle Zahl
Code
Ablöseblase
Projektive Ebene
Leistung <Physik>
Signifikanztest
Berline
Perspektive
EDV-Beratung
Vorlesung/Konferenz
Projektive Ebene
Physikalisches System
Computeranimation
Übergang
CDN-Netzwerk
Schnittstelle
Umsetzung <Informatik>
Gewichtete Summe
Quader
Selbst organisierendes System
Mathematisierung
Computeranimation
W3C-Standard
Metropolitan area network
Benutzerbeteiligung
Standardabweichung
Code
Pi <Zahl>
Zusammenhängender Graph
Softwareentwickler
Schnittstelle
Caching
Einfach zusammenhängender Raum
Automatische Indexierung
Protokoll <Datenverarbeitungssystem>
Plug in
Biprodukt
Automatische Indexierung
Projektive Ebene
Speicherabzug
Information
Portscanner
Metropolitan area network
Automatische Indexierung
Euler-Winkel
Automatische Indexierung
Zoom
Vererbungshierarchie
Vorlesung/Konferenz
Reelle Zahl
Computeranimation
Prozess <Physik>
Euler-Winkel
Versionsverwaltung
Gruppenkeim
Zahlenbereich
Kartesische Koordinaten
Computeranimation
Chatbot
Metropolitan area network
Knotenmenge
Web Services
Softwareentwickler
Haar-Integral
Tropfen
Parametersystem
Automatische Indexierung
Sichtenkonzept
Reihe
Elektronische Publikation
Biprodukt
Zeiger <Informatik>
Partitionsfunktion
Chipkarte
Portscanner
Automatische Indexierung
Reelle Zahl
Versionsverwaltung
Randverteilung
Automatische Indexierung
Metropolitan area network
Bit
Client
Automatische Indexierung
Versionsverwaltung
Pi <Zahl>
Vorlesung/Konferenz
Versionsverwaltung
Default
Computeranimation
Automatische Indexierung
Bit
Open Source
Division
Biprodukt
Computeranimation
Chipkarte
Metropolitan area network
Automatische Indexierung
Vorlesung/Konferenz
Default
Personal Area Network
Fehlermeldung
Punkt
Gewichtete Summe
Oval
Extrempunkt
Login
Whiteboard
Computeranimation
Virtuelle Maschine
Metropolitan area network
Notebook-Computer
Datentyp
Vorlesung/Konferenz
Installation <Informatik>
Softwareentwickler
Varianz
Signifikanztest
Automatische Indexierung
Tropfen
Kategorie <Mathematik>
Division
Systemverwaltung
Biprodukt
Menge
Automatische Indexierung
Rechter Winkel
Reelle Zahl
Instantiierung
Schnittstelle
Server
Punkt
Inferenz <Künstliche Intelligenz>
Systemplattform
Computeranimation
W3C-Standard
Deskriptive Statistik
Metropolitan area network
Informationsmodellierung
Benutzerbeteiligung
Datenreplikation
DoS-Attacke
Softwareentwickler
Bildgebendes Verfahren
Schnittstelle
Trennungsaxiom
Parametersystem
Automatische Indexierung
Division
Singularität <Mathematik>
Portscanner
Bildschirmmaske
Diskrete-Elemente-Methode
Automatische Indexierung
Login
Mereologie
Datenerfassung
Ablöseblase
Server
Reelle Zahl
Information
URL
Versionsverwaltung
Verzeichnisdienst
Instantiierung
Schnittstelle
Server
Gerichteter Graph
Hausdorff-Raum
Information
Extrempunkt
Systemplattform
Simplexverfahren
Signifikanztest
Computeranimation
W3C-Standard
Metropolitan area network
Total <Mathematik>
Softwareentwickler
Softwaretest
Automatische Indexierung
Singularität <Mathematik>
Netzwerkbetriebssystem
Elektronische Publikation
Ausgleichsrechnung
Portscanner
Diskrete-Elemente-Methode
Automatische Indexierung
Strukturgleichungsmodell
ATM
Mereologie
Datenerfassung
Versionsverwaltung
Instantiierung
Data Mining
Binärdaten
Inklusion <Mathematik>
Automatische Indexierung
Data Encryption Standard
Schnittstelle
Server
Subtraktion
Zeiger <Informatik>
Information
Extrempunkt
Simplexverfahren
Signifikanztest
Computeranimation
W3C-Standard
Metropolitan area network
Automatische Indexierung
Total <Mathematik>
Datenerfassung
Versionsverwaltung
Logik höherer Stufe
Informationssystem
Gammafunktion
Tabelle <Informatik>
Data Encryption Standard
Automatische Indexierung
Server
Dualitätstheorie
Desintegration <Mathematik>
Singularität <Mathematik>
Benutzerhandbuch
Binder <Informatik>
Zeiger <Informatik>
Verbiegung
Signifikanztest
Computeranimation
Integral
Portscanner
Metropolitan area network
Diskrete-Elemente-Methode
Automatische Indexierung
Mereologie
DoS-Attacke
Schnittstelle
Bit
Gruppenkeim
Zellularer Automat
Kerr-Lösung
Computeranimation
W3C-Standard
Metropolitan area network
Virtuelle Maschine
Client
RFID
Wrapper <Programmierung>
Zusammenhängender Graph
Installation <Informatik>
Softwareentwickler
Speicher <Informatik>
Konfigurationsraum
Schnittstelle
Signifikanztest
Automatische Indexierung
Nichtlinearer Operator
Digitales Zertifikat
Systemverwaltung
Elektronische Publikation
Biprodukt
Diskrete-Elemente-Methode
Chiffrierung
Automatische Indexierung
ATM
Client
Datenerfassung
Server
Authentifikation
Bildschirmsymbol
Information
Versionsverwaltung
SIMA-Dialogverfahren
Softwaretest
Signifikanztest
Resultante
Schnittstelle
Server
Bit
Subtraktion
Einfache Genauigkeit
Mailing-Liste
Extrempunkt
Biprodukt
Elektronische Publikation
Term
Systemplattform
Signifikanztest
Computeranimation
Rechenschieber
Metropolitan area network
Virtuelle Maschine
Automatische Indexierung
Prozess <Informatik>
Total <Mathematik>
Pi <Zahl>
Vorlesung/Konferenz
Datenfluss
Softwareentwickler
Verzeichnisdienst
Gerade
Gammafunktion
Signifikanztest
Server
Konfiguration <Informatik>
Template
Plot <Graphische Darstellung>
Baumechanik
Information
Web-Seite
Elektronische Publikation
Natürliche Sprache
Signifikanztest
Raum-Zeit
Computeranimation
Metropolitan area network
Client
Menge
Server
Information
Konfigurationsraum
Wurm <Informatik>
Softwaretest
Metropolitan area network
Bit
Automatische Indexierung
Test-First-Ansatz
EDV-Beratung
Vorlesung/Konferenz
Physikalisches System
Softwareentwickler
Datenstruktur
Computeranimation
Integral
Resultante
Offene Menge
Selbst organisierendes System
Program Slicing
Gruppenkeim
Implementierung
Computeranimation
Richtung
Metropolitan area network
Client
Domain-Name
Pi <Zahl>
Vorlesung/Konferenz
E-Mail
Softwaretest
Signifikanztest
Ereignisdatenanalyse
Content Syndication
Open Source
Elektronische Publikation
Natürliche Sprache
Integral
Konfiguration <Informatik>
Entscheidungstheorie
Arithmetisches Mittel
Rechter Winkel
Automatische Indexierung
Mereologie
Server
Information
Verzeichnisdienst
Instantiierung
Bit
Gebäude <Mathematik>
Plug in
Physikalisches System
Elektronische Publikation
Raum-Zeit
Ereignishorizont
Integral
Reelle Zahl
Automatische Indexierung
Prozess <Informatik>
Vorlesung/Konferenz
Garbentheorie
Softwareentwickler
Leistung <Physik>

Metadaten

Formale Metadaten

Titel Packaging and testing with devpi and tox
Serientitel EuroPython 2014
Teil 12
Anzahl der Teile 120
Autor Krekel, Holger
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/19981
Herausgeber EuroPython
Erscheinungsjahr 2014
Sprache Englisch
Produktionsort Berlin

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract holger krekel - packaging and testing with devpi and tox This talk discusses good ways to organise packaging and testing for Python projects. It walks through a per-company and an open source scenario and explains how to best use the "devpi-server" and "tox" for making sure you are delivering good and well tested and documented packages. As time permits, we also discuss in-development features such as real-time mirroring and search. ----- The talk discusses the following tools: - devpi-server for running an in-house or per-laptop python package server - inheritance between package indexes and from pypi.python.org public packages - the "devpi" client tool for uploading docs and running tests - running of tests through tox - summary view with two work flows: open source releases and in-house per-company developments - roadmap and in-development features of devpi and tox
Schlagwörter EuroPython Conference
EP 2014
EuroPython 2014

Ähnliche Filme

Loading...