Bestand wählen

Packaging in packaging: dh-virtualenv

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Erkannte Entitäten
OK no Europe any passing conference is complete without the least 1 token packaging and this must be released 2nd
so we can see what you're working has to tell us thank you but if you don't mind I'm being wait for a tiny bit for quot foreign just depression
no income actually before your thought of it is your chance of that happening start
thinking it OK buttons on the welcome all has common that that's my outcome of 2 years of German studies what elements to school unfortunately I can't do better by welcoming 1 and so it's all about the tool called the age were joined the board as labeling packaging in a packaging not before we look into the deeper let me introduce myself so my name is beautiful and I am from
Finland but I'm living in Stockholm Sweden nowadays I work for a music streaming company was modified in their eyes do kind of to start society and the convention here so I built like a pipeline of new music service and on the other hand also FIL around a lot with our internal python 2nd answer to to this question about Python if you want to reach out to me
that's my e-mail address and there's much we don't know so we still the and questions now are about this talk this so will be in 3 different
sections 1st we're gonna look into some of the existing deployments strategies you can use are made that that gave them the best machines and all into 1 actually this year Treasury environment and how this differs from the existing deployment strategies and in the end we're going to go through an example of how you itself and with a packets century mostly because it's not a simple piece of software so I'm going show you
example of how you can use it which I to pick something like century production now let's start with that
kind of who here on this page again even the best system this year from correct told them that who
here deploy staff using the so-called native packages on that in a system like relying on the medium libraries on the system who here is kind of frustrated with that as in the case here uses words environment like the set up a mom and then you get installed everything in the same fraction of the no
this both have their good sides to if we take for example the data that packages so this table the stuff that gets in the and especially the stuff that's in name is stable and well tested people usually don't change them so that the backwards compatibility outbreaks so if you're the consistent has opted for for let's
say my friend requests you know that that request something that's packaging covered compatible with the previous version of Debian has enough for that
and packaging is another nice both sides you can declare non-Python dependencies so say your software requires see polite to be installed for your software requires my so it could be on the same machine when he directed the competitors against
that your packets depends on my sequel and at that point so of the machine everything Mexican containing 1 package it also
has really need the existing infrastructure so not only have like it's dedicated to tools or separated built environments like this field another solutions but you also have the possibility of running your own nets repository which means that you will have your own way below right on network of deploying your stuff and production loss lost all kinds of CI tools like Jenkins would have at least some agreement to support for dealing with that impact packages and the last thing I think in the end packaging is is that you have a quite nice support what that means is that if you need to
remove a cash on you up operator your package worse and you can write a script that before you operative packet you think there are to catch for human restorative services you install it you can write a script studies after installation little restorative services you can do crazy borderline stupid stuff like that database migration postings I don't necessary recommend them but you can do lot of stuff there it gives you final problems it was a good part and it was talking about the bad parts you probably have run in this case where you see that let's say back in the days can have worked hard and tossed out request 1 . 3 and that has just a feature you need but unfortunately your ancient a cop would field box is running respond to and what do you think the don't wait for a decade the tactics what happens is
just rope in front of the computer is waiting for the new requests especially if it's a backward compatible to come under the current system is running so a lot of stuff and didn't even at the release of that particularly the variable the release is already outdated and the packaging
itself yes kind of complex so what we're talking here is that someone created detection system that is built for building a full operating system so it covers all the possible cases all the possible scenarios if you wanted to put a girl or Pasco Python you the name in it the test everything covered that means that the whole system really
really complex and the documentation is far from simple analysis that all the information that the user can find is also geared towards
DPM package maintainers OK you wanna ship this thing within the operating system so this is how we do and it does not necessarily resonate how you yourself and we deploy your
time service in the host and what I think is the worst part is that in practice is that you get a global states so if you're only deploying libraries as different packages and then using them eventually you end up having a case where you would like to upgrade 1 library but it's also used by some other software on the same box and you don't know if you can do it without breaking energy we've had this it's 1 those of of different times when we have rolled out like a new common Python you libraries and the posts and you're just to afraid of deploying that's using might we are kind of great that's something I don't I might break you we have been testing for months so it's kind of slows down and it's really really annoying no if you think about your genome alignments that so would you get in the water environment is somewhat the opposite so you get the a whole new stuff we just
to keep installed and you get whatever available like I like the latest release can and you can go you go to the extent of light you can
open you did mercury and for the stuff in your environment and from there so you can always get the news stock but it has become a de-facto method and the python world so every guy guide usually contains of water to how you want something so there were 2 you get the same water and wine stuff when you laptop as it into a servers I've kind of works but it's also battle-tested so know there's so many people running production so it's fairly safe to use that 1 but the best part I think is that is contained so if you take a package and so inside word environment a Python package you can be sure that it won't affect anything outside the environment so updating a simple packages over doesn't mean that your whole system crumbles because something was relying on all this on older versions for w interferes with the underlying operating system know you're only poking their actual water environment and you get this nice contained forcibly did not now the downside I bet some of you might have seen this slide that means that you can't have any native dependencies if you're using that were environments install you need to know what my sequel libraries you have to to have available like in this case you need to know that you have been some icicle client to find the mystical conflict by evolves so it's advise you do some manual the kind things you know more than you can do source stuff OK so you can have wheels or even that what sources but if you don't have wheels for a platform or you you don't have to have instead of your own we will register you end up doing source installations and we have reworked environment in your production servers you end up installing all the dependencies for that source installation on the production environment which is not necessarily bad it's that you break something but you just gotta you fraction environment and development areas and other you should uh and unused files for the most part what I think is with the the principle is that you're basically executing a bunch of random scripts so nature we'll get around this but if you run set up fight installed you probably haven't looked into what all those spots do or what all those files that those that it is dependent on so you're just relying on good faith of of people and European like running random stuff in your production environment it doesn't need to be malicious the Virtue someone might just accidentally released effective white your whole UTC audible director something and you by accident linear system no this brings us to 1st then what is that the so far did for German was about so 2 years ago my attempt to combine the best of the 2 worlds so it is the words and environment that is placed inside the package it supports both sides and you have read it is kind of person agnostic I want become used by the ones that with that but it doesn't execute importing and it doesn't mean important Python code so it doesn't really care about if you're 2 was written by 2 or by the 3 you can use it anyway it even supports using their new were trying environment package with 3 by 3 so you don't you need to install water and to run it's also source so the GPL like all that can build tools it has a good the conditional I'm the guy who wrote so I have my David dies but I think that the communication is to is that is better than the average of this is the condition the best part which seems to be actually very functional is that it has a simple tutorial so if you go to engage for term that we don't support you find this of course tutorial that you can run through and boom you practices inside inside of water environment inside the package Our under the surface and it is a bit of extension so they're number is How would have been made pretty like a pile of perspective that that can executed Our when you're building packages it's a certain fixed sequence of 1st groups that you install and there are different extensions for that to know how to how the good practice for python packages of particle proclitics is more valuable unabashed provisions stuff so what did for China and does just injects itself that fall so this 12 lines parent included there's that there's that magic being injected if I were generated there and then it just France as part of the sequence now this is kind of like the implementation details about for you already have existing given that and build environments like
if you use as field or something else or just plain the bill for peta-scale package from the command line this means that using the average time just gonna word existing work so what basically they did back in the days was that I found a great blog post by
the next level I adapted the idea that it's to fit our built environment and that's why I ended up writing the proper extensions and next system works of well to produce a thing called which back in the days didn't fit our system no in practice but it worked and this affected build that it's trace was wasn't where till environment you can define what Python want use with it so you have multiplies and in machine you just pick the 1 you want it installs everything you have with the requirements of the the and this is the exact same format together with the phrase that it is cells those inside the word environment then takes your project and run set up the time slot on that so it just doesn't Dr. resources and they're actually installs your project inside women and then it doesn't want magic which is like sets with word defects actually goes to
connect those and other stuff like free writing activity and so so that you can actually run like instead of having all your bills system has to continue production system has and you can use like things like activity directed at this in the production end up with the same water environment OK so that's classical but let's see let's say have projects let's packets something this return let's take it's central so who here knows what centuries but that it's a really good exception track into
we use our production systems and it works like a charm are the best part for example part is that when you install century if you have ever done it is so central pulls down like half the Python Package Index is like every can find depends on a lot of stuff is not because of that software it is because complex
authoring in that way decided that it is but anyway let's let's see how we do the important so 1st thing is that we need install the if you're running a modern operating system like you going to trust or the they and testing the data for time is actually available inside the frequentist again just have it installed literature and and all of a sudden you have this return did trying available in some a system it's as previously discussed in the that the UN and you going to but you know it still works then you need to create a dip in directory inside your center installation so the bin directory is a constant vector that induces the figure out what what stuff should not like it from that the use of what should build what substantive run build time and in that directory you have to create a few files this minimum set of 4 from positive to create the and don't be afraid all of these are covered in tutorials
so so when he start senior essentially you add a controlled by
now this is the place with which depends figure out what this means for building so you can see that century requires that Python development is for building but for running it doesn't require anything special so in this place you have to just below dependent it was and then right the required dependencies like Python something that there but this is basically what how I did this today when I that essentially the is that I just copied order tutorial stuff and change the field that I felt the need to be changed by Pakistan's needed change what the change that is what to
change what understood 5 that required for the impact figure out some version and the cells that then you're Pegasus inches 6 . 4 . 4 called and the 3rd 1 this is like the is called the complex that
matches the needs that that when you need to define what is called a competitive level so that the pianos what how which built the package what's relevant to you guys probably just yeah combining them in this complex valued and if you don't do that you pick up some ancient embedded level and will you package so that's rejected and the last part is
the glorified Make file rules by which just tells you now if
you build that the impact before for Python you recognize you probably recognize final again to see that we change the title to 2 the trying this dispersive does that the end this packages engagement instead of the default way of building height and stuff and that's it if I the package go package and also rose to you get nice matrix like output
of stuff building and all of a sudden you see
look that it's actually pretty words environment puts it what not in the best cooling cost Internet down into a package and that's done always left for you it's
just take button and it's copy the production post and still in there
now if you had to find something and something that has some went dependencies in the control file they get installed at the same time and the best part is that you haven't executed in a random street and of production system because all of them were done in your build system you and you end up with the deploying the whole thing without cluttering any but in practice system with any development assistance fact that and you have a nice contain rich environment in your direction OK so once you've done that let's look at the Cannes Nice parts of the data which are and then so what it gives you is that it gives the non-Python dependencies for the possibility that defines the Python dependencies just way you could do with that normal their temperatures at also leverages on the existing infrastructure of that in building so you can use your existing village you can use the existing CS systems already using that would that be and where you can have your own triple still in use it has a new wilderness so it's not come to you it's not limited by what you can find that the and that you can actually just to install what's the use of available and as I told before its contained so we end up with the word to environment in certain places near correction system and that's it of course like like in solution there are also some negative points there besides this the bill times can be slow so especially if you're not on your own like Europe and you're not using wheels you're basically downloading all the requirements from the Internet and the building them which means that yes time will become along with a longer but it can be also substantially mitigated by running on mirror which cuts down then it's latency and using we use which cover cost of times but it still requires you did some clients so you need to know what requirements you need what needs system requirements you have the systems in the sense that you are that all so if you're running let's say you're passing XML using an XML you need to make sure that your control file dependent depends on installing the ammonia practices as as having the development in response to the the build system needs to have the exact same so light because it's works environment what water was and thus it actually
links outside which environment for the while Clinton links stuff from the purchase system so you need to have the same Python available on your build system Macedonia production system but it varies from problem if you are already having an existing the lessons that Fitch and stuff but for the future of the return and as a stranger to solve this stuff like I'm looking into cookie-cutter you gotta templates unfortunately I don't have time to prepare them for before this talk but it this week if you could just use the culture and boom you would have units which are targeting down with the without you don't need to go and call random forests no and buying that occur support like if you get a minor update of reply from 1 neuron traction hosted refreshes the words environment matrix once and also this was actually from terms of that made from Adam from this this when I want you to reversing this talk we talk about this and he came into play what if I could actually break out that depends in system by like use or something to incorporate the whole place them into the evidence environment but that's pretty much it did you have found
out more the source available open-source
on this but if wrong In gets up and there there's the good emphasis on the word good documentation on the redox and then the support oppose posted 1 released this so With that I thank you for your time be thank you can you just
males have time but any
questions to the microphone higher the grade cells thanks very much for that uh the the uh the time I was looking at your presentation at your but your colleague from Spotify about using doctor to the for the deployment and the managing the infrastructure I myself am right now struggling between Golkar for their and production and right now in my company we are building the today again packages actually so this would be really really cool to use uh do you actually a dual there that you install software based on this the poem was that in pekingese and some other deployments based on docker or do you mix it our that current plans
Microsoft artist makes so our point while like while doctors great it still doesn't solve the problem you are defined and this is your so it basically boils down to 2 options so you write the doctor was it the rest of the year that says like this became so that after this these which can be tested it but they have the same problem that you would probably like to different document is 1 for building your package somehow that extracting out of putting into the other dropping during that so the benefit of the operated in the city to which our environment the doctor already provides isolation but clearly more more words to Billy Pizer of this and then use the CAM packets to dump that went into the images a uh and the other my other question is what do you use for your local amplitude was I have no idea but something fairly off the shelf limited but it's like yeah ancient installation it's a good question does that support great weekend character that the person with those of through stemming like kind of a great weekend this if you want to I don't like to think that temporal that you database in its immigration and you wonder package to be able to to some me some I don't know whether were table for it you know your packaging we need to have some changes from 1 region to another so because it's it's so that in practice so it will have all the posts as which and stuff like that they can run so if you want to run the streets before removal after removal before installation and so forth you can use the existing infrastructure for that 1 so it's not it's not too complicated to those for the person who has some knowledge of the that impact in that yes the uh 1 question related to that and to use these imports insulation translation process database migrations students that you use something different but what use the owl we do have we actually do have some form of project and use the post fast afforded migration but it's kind of as you status theory like something tickets of and we get like an unwanted migration in the test set so you see we do that and that was like this with some sort of manual depending on the project for the and you can do postings at the end of the project has been with finished 1st with can you use this system Python packages are overworked everything is start to to be in the 3rd and 4th example it elecciones and below in managing library yeah currently by design everything is installed in the 1 so but it shouldn't be like it's not it shouldn't be too hard to add like feature did for and that we should you both legs at the same time of system packages to be busy yeah as you're uh I see your point is of a similar to low art kind of annoying while it's values of pillars reanimating stories and pretend once so this was something for future that fact that if you want to use the site packages the existing ones and why not what kind of maybe you can see them all the problems requirements 60 and just to depends there yeah but then like he added depends it gets is on the system level but there were 2 environment is filled with there's no site-packages flight so it won't see them 1 1 like that intermediate that you could take is that you could depend on the well but that is on the system level and then you could use Python path when you start here salsa at point that in extending uh in addition to that over 10 runs by them that's so that would work but I wouldn't go for I want to assure its why was solution but nevertheless like if you wanted to each of these the ticket and again it should be fairly simple they site and just below that thank you or if you want make read is even better the about PM and so called the package management yeah so that's the the next book posts which is actually like doing this exact same stuff but using at the end is instead of injecting the purpose sequence a 1 the use of originally it works and gets the job done but it didn't fit the bill systems and so then I ended up building and that for sequences that OK but it's someone to start from scratch and doesn't and you can see is that the you will want to use that then the how and why he we could use your product instead of the and the advantages of the but I don't know if there's a this is reason for like that in that case of say sago retinex proposed check of the pixel and the imitation of my project and like this I don't know which 1 is inferred so by indicator people don't really know that in packaging and and 2 so should be used but it's like the same kind of property just follow steps and of and so it's not a matter of personal preference like very much
but the fact the just a quick 1 is there and plan slot to to we see for example using that holds but the mental yeah that's that could be done I'm already planning to like because trustees . 6 so I'm planning of this set of gave states of it again pick the new releases on trust the this stuff you find this is about building a music but we also already stable at that point when I released our so it shouldn't be too hard to add that works for research but is still than all technical reasons not to the notion of what you know this is this is really simple and right We we build stuff with the torture and the please so it so if it doesn't work on with it and we're done something really bad about half this of the and thank you Nestle's questions thank you learn from various in very useful tool in the
Schreiben <Datenverarbeitung>
Geschlossenes System
Geometrische Frustration
Gruppe <Mathematik>
Translation <Mathematik>
Skript <Programm>
Elektronischer Programmführer
Analytische Fortsetzung
Kategorie <Mathematik>
Gebäude <Mathematik>
Güte der Anpassung
Dienst <Informatik>
Wurzel <Mathematik>
Rechter Winkel
Zellularer Automat
Ordnung <Mathematik>
Lesen <Datenverarbeitung>
Tabelle <Informatik>
Folge <Mathematik>
Baum <Mathematik>
Automatische Handlungsplanung
Paderborn Institute for Scientific Computation
Open Source
Virtuelle Maschine
Weg <Topologie>
Migration <Informatik>
Endogene Variable
Virtuelle Realität
Skript <Programm>
Installation <Informatik>
Amenable Gruppe
Architektur <Informatik>
Dispersion <Welle>
Open Source
Binder <Informatik>
Elektronische Publikation
EINKAUF <Programm>
Komplex <Algebra>
Wort <Informatik>
Eigentliche Abbildung
Prozess <Physik>
Web log
Natürliche Zahl
Fortsetzung <Mathematik>
Element <Mathematik>
Komplex <Algebra>
Einheit <Mathematik>
Prozess <Informatik>
Figurierte Zahl
Funktion <Mathematik>
Nichtlinearer Operator
Installation <Informatik>
Klassische Physik
Konfiguration <Informatik>
Arithmetisches Mittel
Strategisches Spiel
Projektive Ebene
Web Site
Gewicht <Mathematik>
Zellularer Automat
Kombinatorische Gruppentheorie
Physikalische Theorie
Physikalisches System
Bildgebendes Verfahren
Elektronische Publikation
Wald <Graphentheorie>
Cookie <Internet>
Physikalisches System


Formale Metadaten

Titel Packaging in packaging: dh-virtualenv
Serientitel EuroPython 2014
Teil 58
Anzahl der Teile 120
Autor Pulliainen, Jyrki
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/19992
Herausgeber EuroPython
Erscheinungsjahr 2014
Sprache Englisch
Produktionsort Berlin

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Jyrki Pulliainen - Packaging in packaging: dh-virtualenv Deploying your software can become a tricky task, regardless of the language. In the spirit of the Python conferences, every conference needs at least one packaging talk. This talk is about dh-virtualenv. It's a Python packaging tool aimed for Debian-based systems and for deployment flows that already take advantage of Debian packaging with Python virtualenvs ----- [Dh-virtualenv] is an open source tool developed at Spotify. We use it to ease deploying our Python software to production. We built dh-virtualenv as a tool that fits our existing continuous integration flow with a dedicated sbuild server. As we were already packaging software in Debian packages, the aim of dh-virtualenv was to make transition to virtualenv based installations as smooth as possible. This talk covers how you can use dh-virtualenv to help you deploy your software to production, where you are already running a Debian-based system, such as Ubuntu, and what are the advantages and disadvantages of the approach over other existing and popular techniques. We will discuss the deploying as a problem in general, look into building a dh-vritualenv-backed package, and in the end, look into how dh-virtualenv was actually made. Goal is that after this presentation you know how to make your Debian/Ubuntu deployments easier! [dh-virtualenv] if fully open sourced, production tested software, licensed under GPLv2+ and available in Debian testing and unstable. More information of it is also available in our [blogpost]. Talk outline: 1. Introduction & overview (3min) * Who am I? * Why am I fiddling with Python packaging? * What do you get out of this talk? 2. Different shortcomings of Python deployments (5min) * Native system packages * Virtualenv based installations * Containers, virtual machine images 3. dh-virtualenv (10 min) * What is dh-virtualenv? * Thought behind dh-virtualenv * Advantages over others * Requirements for your deployment flow * Short intro to packaging Sentry with dh-virtualenv 4. How is it built? (10 min) * Debian package building flow primer * How dh-virtualenv fits that flow * What does it do build time and why?
Schlagwörter EuroPython Conference
EP 2014
EuroPython 2014

Zugehöriges Material

Ähnliche Filme