Real virtual environments without virtualenv

Video thumbnail (Frame 0) Video thumbnail (Frame 4374) Video thumbnail (Frame 5531) Video thumbnail (Frame 6549) Video thumbnail (Frame 10093) Video thumbnail (Frame 11564) Video thumbnail (Frame 12118) Video thumbnail (Frame 22589) Video thumbnail (Frame 23420) Video thumbnail (Frame 23936) Video thumbnail (Frame 29408) Video thumbnail (Frame 30456)
Video in TIB AV-Portal: Real virtual environments without virtualenv

Formal Metadata

Real virtual environments without virtualenv
Title of Series
Part Number
Number of Parts
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Mihai Iachimovschi - Real virtual environments without virtualenv Virtualenv is a great tool for the development environment but it's definitely not suitable for every use case. Also, Docker is great for running the application in production, but not everyone that use it in production tried to use it in the development environment. Why not use the same tool from the beginning of the project and until it hits the production in a uniform stack of tooling? This talk will show use cases of using Docker in the process of development as well. ----- The process of developing using Python is very straightforward and easy. Still, each and every developer has his own style of developing and building his entire dev environment. Most of us use virtualenvs which are reliable and comfortable to use. But there are some issues. For instance, the repeatability and immutability of the built environment are not guaranteed. Virtualenv does a lot of work that targets the direction of somehow isolated and independent environments. They are *almost* *fully* repeatable. In any team, we can hear the notorious expression "It works for me!". For some time now, I am using Docker instead of virtualenv for building custom and really-virtual environments that are entirely isolated. The containers are immutable and consistent, so this workflow guarantees repeatability. Using such technique, not only enables the user to have unique and immutable environments, it also allows de developer to create full app architecture that can then be tested and deployed as is. So the production version will be in identical conditions as the one from the development environment. These features are not provided by virtualenv at all. The goal of this exercise is to try to use totally different tooling for building the application from its first line of code until the production.
Point (geometry) Presentation of a group Service (economics) Overhead (computing) 1 (number) Perspective (visual) Product (business) Revision control Thermodynamisches System Computer configuration Different (Kate Ryan album) Operating system Computing platform Distribution (mathematics) Wechselseitige Information Server (computing) Software developer Consistency Projective plane Parity (mathematics) Variable (mathematics) Computer animation Integrated development environment Multi-agent system Internet service provider Network topology Order (biology) Lipschitz-Stetigkeit Virtual reality Library (computing)
Standard deviation Consistency Software developer Binary code Sampling (statistics) Control flow Cartesian coordinate system Virtual machine Product (business) Subset Formal language Computer animation Thermodynamisches System Virtual reality Resultant
Point (geometry) Group action Connectivity (graph theory) Computer-generated imagery Virtual machine Mereology Medical imaging Flow separation Thermodynamisches System Operating system File system Damping Selectivity (electronic) Process (computing) Endliche Modelltheorie Error message Pairwise comparison Namespace Content (media) Shared memory Basis <Mathematik> Cartesian coordinate system Portable communications device Process (computing) Kernel (computing) Computer animation Visualization (computer graphics) Network topology Summierbarkeit Library (computing)
Addition Multiplication Existence Computer file View (database) Multiplication sign Device driver Directory service Arm Cumulant Medical imaging Computer animation Thermodynamisches System Read-only memory File system Office suite Data structure Summierbarkeit
Context awareness Installation art Multiplication sign Source code Workstation <Musikinstrument> Execution unit 1 (number) Primitive (album) Parameter (computer programming) Neuroinformatik Medical imaging Mathematics Coefficient of determination Blog Single-precision floating-point format Mapping Building Computer file Moment (mathematics) Sound effect Instance (computer science) Demoscene Arithmetic mean Process (computing) Order (biology) Quicksort Ranking Cycle (graph theory) Volume Game controller Computer file Real number Computer-generated imagery Artificial neural network Virtual machine Streaming media Login Number Product (business) Revision control Natural number Gastropod shell Mobile app Default (computer science) Standard deviation Stapeldatei Server (computing) Weight Projective plane State of matter Content (media) Code Independence (probability theory) Coma Berenices Volume (thermodynamics) Directory service Line (geometry) Cartesian coordinate system System call CAN bus Word Computer animation Integrated development environment Personal digital assistant Blog Statement (computer science) Local ring Library (computing)
Complex (psychology) Latent heat Data management Service (economics) Computer animation Integrated development environment Thermodynamisches System Different (Kate Ryan album) Coordinate system Product (business)
Run time (program lifecycle phase) System administrator Decision theory Mereology Web 2.0 Medical imaging Cuboid Position operator Social class Service (economics) Virtualization Instance (computer science) Variable (mathematics) Web application Type theory Configuration space Normal (geometry) Quicksort Volume Slide rule Service (economics) Overhead (computing) Computer file Virtual machine Device driver Student's t-test Hypercube Field (computer science) Product (business) Revision control Latent heat Thermodynamisches System Energy level Task (computing) Inheritance (object-oriented programming) Content (media) Coma Berenices Login Database Device driver Grass (card game) Cartesian coordinate system Mathematics Word Computer animation Integrated development environment Visualization (computer graphics) Password Moment <Mathematik> Window
Building Scheduling (computing) Theory of relativity Multiplication sign Software developer Characteristic polynomial Term (mathematics) Cartesian coordinate system Product (business) Revision control Medical imaging Thermodynamisches System Kernel (computing) Computer animation Integrated development environment Thermodynamisches System Different (Kate Ryan album) Personal digital assistant Order (biology) Dew point Endliche Modelltheorie Window
they had legacy can get easier the 1 to me how wrong and I think it's all for the unions so I also reveal some of my personal experience regarding the mutual development environments so let's take a look at the main key points of this presentation so the person is consistency or value development environments should be similar in the actual production environment in the development of how about their diversity warehouse Europe you should handle did that distinct environments for existing products the isolation work on different environments should be different from different projects should not overlap and great each other and so in the beginning let's see how they usually people like Python developers are of doing their work for so in order to have all of the dependency of locally used people were they always packaging so all installing pre-compiled packages from the operating system picture poster what even it is useful to have all of them locally but how do you delete so do you do you have to be a solid European globally work is the use of condoms or mutual and for any other similar tools were work prefectural there are the last 1 eventually quite and comes to consistency but otherwise it is in the overhead this and have finally your environment up in your version control and policing feature official development of the deployed well so you can definitely multiple options what you can do it on a platform as a service provider or unknown its infrastructures services service people it and even deployed on the also release anorexics those 2 are quite similar ones from the deployment perspective although you can fill in the rest of ethics so a virtual lands against our really really popular knowledge there is a kind of publishing variables were would find the super simple there global the showed system libraries and packages a but the pure is a small problem because running on the shirt system libraries that you cannot nucleons might be a problem but these underlying leaves are are different in front of the distributions of well some of the distributions try to be on the bleeding edge other distributions are more conservative and concentrated on the stable packages also so even though it's possible to specify all of their specific version of the internal Python dependences on inside the virtual environment frequently dependency trees not only bond ladies internal Python dependencies but also depends on the system-wide his lips that brings inconsistency between
different missions moreover this inconsistency might things subset of production to all when a given system dependency of the 1st in some manner from with the 1 from the development of mission so whether something breaks there's always means that there is excuse so imagine for a 2nd that we can have the exact same thing but in all the the environments in which applications running you know when write in Python which is an interpreted language which results are not compelled into some kind of self-contained of binary so let's containerized all Beckett of something of more robust than the single virtual environment so which would build the kind of binary with all of its the is enclosed in the was is in a
container so as the doctor became an industry standard for the containerisation always samples will be based on the docket acknowledges what the passing this
talk at this point I've got some questions like all the presidential machine uh basis and I was as follows different from other ritualization rejoicing technologists in fact of visual and you sort you in fact containers are not official machines their new there is generally likely visual mission they're just a process the trans under the same kernel the process is just the isolated and it's running in its own namespace
good so of you can see the same of pictures before but using the beginning of it so we can engine running around all of the shirt system libraries and packages and that and then the sum of all the different and that and steal it somehow similar to the social and situation but all of the different thing is the fact that contain errors are just self sufficient because of all of this is that all of the dependence is inside there's so they're portable because there are more more almost so let's talk about the anatomy and basically components combined just the 2 features of the more the Linux kernel of those artists groups and namespaces so see groups are enable users to prioritize and we made the system resources for a specified process that means this is offer comparable with the method of selection would of course completely isolates and applications you on the operating system so we it includes process trees networking user reviews among themselves systems so let's see what is actually inside a container contains all the system dependencies all libraries and has limited and points for access to and from the outside wall of the container is built from an image which is immutable 1 of the images are self-contained before they are I think I think New you for these ecution model inside they are isolated so the different images I don't share any parts of them and they're immutable within basically stays in and of course but they're portable so you can move them to any machine and they will work just fine I speak about something out of the container is basically just an isolated process that is disposable as well so integrating content and use of the varied and start from scratch from the same image the content of the running over the image file system layers which is a your 1st so everything happens to be in the container is just a layer on top of the image so let's take a look all
the office works in your favor since the unification file system means that it takes multiple directories in a single women cause stakes them on top of each other and provides a single unified view on them In addition this a office uses the union and that's how the images are immutable so the running container is just the layer on top of the layers that are played read against it and that's how containers are disposable so I think it was in this layer without being committed to the actual image of a volatile because it's this poles of the time when the container is removed without altering the structure of the image but notice that multiple file-system directories on you can use that we can just add or override fast because it's cumulative so how to actually do we define we don't simply writing it with another 1 to achieve these the
union fell system driver just places a DIY colored filed in the container of filled public so the white file we went out file just effectively obscures the existence of the files in the read-only image layers below so the file is basically still there is just not visible to the user but the where you
and all of them you can delete the images you you can do it filed with with white out is for for remembering that it's still cummulative and the size of the image will only increase so that can the imaging unnecessary files will end up getting huge files for natural reasons OK so what up to this moment it seems that all the these mutable the players in the image are read-only so how actually to work with the applicant real application via the following we we will actually use for these problems that among these external resources to the content alone will basically is this external thing you can just monte there the from the course the directly to the context and the volumes are not anyhow managed by the doctor so there never a cycle or the doctor never do it's them when you and remove a container the initial now that we know all the primitives of let's see how the images that actually build so for that will need just 2 things the 1st 1 will be the raw product or the source for the for applications and the next 1 will be the computer science which is represented by the fact so but that's basically called super simple the file looks like a and would do it every line from the local file represents a layer in that actually image of newly built and let's go just line-by-line and try to get a deeper look at this so the 1st line represents the base image of the which will be 0 our only image it'll it'll take this image from the problem it's a sort of new call for the of images but then we set in environment variable instead of our content then the rank among just basically runs the commands applied to the this command so will just create the director for our culture and then we have this specify specified file to this directory and so of afterwards you can just run consolidated and get all the local but independence is the inside the image then we simply of the cold inside of the scene you may actually wonder why did get combined the 4 4 and this lines of having like I think all the cold of before being the biggest all but actually there is a reason for that is because the potential the layers so if nothing is changed it will just to use the the delay the lives of the image from cash so when we change the cold it shouldn't use you just reinstalled in doing this because delta region over time so we know that if you to profile as we build on image from it and would run it also will take a look and how to get the logs from and uh running from the and we will try to sneak inside some of containers to together even more control so let's begin with that building the images which is defined by the 1st argument but then specify the image that which is basically the image name and we can even found a colony of some of the number of we and other than that word than a number which will define the version of this image but by default by the full this suffix uses the word weights and then we specify of these dogs which it tells local to search all for the local file in this particular directory if you want to show the built command outside from this project from the project directory you should specify the full path to you know which the problem is leaving and now that we can the imaging place let's run it of the 1st step should be to sort self-explanatory that then we just have map to the port from the content from the running container to the port from the cost of what would the demand is right but then we specify the imagination of the spot before the bill stage so make sure that we run from the specific image than just around the usual command for running a different instances gender of in this case all the application evokes to go to to this the old of now we found to run this as a statement is the text so we can just specify with this of that of that which is that's and in order to be able to identify this container easier from other continents live you name and I was mentioning before about the problems so let's actually mom according to the container for we can make changes to to the AP on the fly when they're machine well these all and this is how it's done and well this someone works but it's quite a mess will come back to it later so as mentioned before get in blogs so when you run in the batch of the logs are not you're going to use the standard LocalTalk soul so you can grappled with the local export of minus that is basically the same have the same effect as the minus an argument from the good tell command which follows the stream and here is the use of the name of our content and if you need for any reason to get inside the run container you can just execute shell inside a unit with local exact minus safety means that it will be interactive and it will allocate Citigroup's suited to the light of forward to be able to interact with the shell then it's followers of by the name of the container the and then was specified this show that 1 from but the were usually the base images don't have inside of them of sensor shall then try run from the shore this age or other analysts of single isn't installed it to the image also using the exact command and just execute any other commands the inside of the container so that the process is totally isolated from the other processes on the same workstation because all the specific dependence is bundled inside and these libraries and means can be distinct from the ones that are installed on a system-wide on then of course that book engine is running uh and it's disposable so whether something goes wrong or you can just stop there with a specific container and start again from the same image but
there's although some of them not you may wonder how about a distinct external dependencies and where the promise that's like environment well here is will just need more containers an and that's how
production environment should look like with different containers all of them with the specific goal with vendors civic and working together in a wonderful synergy if
they were universal achieve this synergy will need of some kind of orchestrate the recession is that automated arrangement coordination and management of complex interdependent systems and services I should this simple will consider different approaches for production and their environments but the
simplest way they used to to do this with of composer and and this is the following so 1st simple political folks like and can see that I defined the 2 services on therefore the first one is the database using the stock of positive image from the skull the 2nd 1 is the web application which were building before and as you can see it because all of the variables that of that were defined for mentally defined heuristically soon and we bring up these 2 containers that with this class is just just as simple and you remember about the awful along command from the this slide before it's just as novel that had some sort that the represented in the config file so use of this and he she wants to use composing brought out you can of course any other uh the environment which is but testing staging or whatever you can create base normal file that's will it could contain all the generic configuration and just extend them for a specific of environment so let's say that the production composed something and this is the use of the Web will contain a of not using them for example admin of running command buttons using you with and depends on engine as well that so running the ball in production of is not the only way of doing the orchestration and is perhaps not the best way place as well I will not go much into the details but let's so do a small overview of the industry and solutions for orchestration for all being as well and for secrets so for what decision production you can use of course composers described before work is locus swarm if you have a swarm of all instances running out and want them to socialize with each other also the external tools and that are good for orchestration like Council on I've also heard people being kept was a capable of never tried souls secrets but there are multiple opinions regarding them but this lesions stored the secrets of the passwords on the other any other UK he's and so in the version control systems with that kind there are different ways of passing secrets to of applications from the environment variables to think of all of the sea grass or any other way at runtime covered 3rd party tools that are designed just for this task like book of all which is built on top the voltage and the week and their their storing the passwords safely into the cost and passes them to the actual running containers runtime and he from using government is or Google content in general you can hold these features built-in thus began bulk-loading starting with version 1 . 6 we have the friends blocking drivers and it can easily use them on is strictly depends on loading the students at the system level uh but as part of the whole school basics you can just use to slow and be kept and
then you have this 1 person uses wind up with for them because there is no torque about as seriously for developing or Windows or Mac you know that you cannot run Dr. natively on the machine but not more to use of official boxes them where bottles or other type of words the virtualization field because there is a new book of remarkable in windows and works leaving with fast but with some marks on marks the user is the user makes 5 which is a super lightweight visualization solutions for almost 10 almost no overhead and the use hybrid well something this
out of this is a nice way of building environments behave in the same way in production and development so it's really easy to make self-contained packages of with all the particular dependence inside them so you can with some exotic the premise is that both not exist in the wild world also having all of them in Europe guarantees that the there are no differences find no there's problems related to this chemical there are also isolating them from from each other just and just explicitly specifying the order interfaces or hallowing particular continues to communicate with other containers that minimizes the risk of unexpected conflicts and all these characteristics improve the overall robustness of the application in relation to its environment so that's basically it thank you as well because of this schedule model just before and you know around this emphasis of this room out OK can we have the time quantity questions and you have a question hi thank you an excellent talk to 5 stars and and I have a question so documents as the commission so that the assailant documents you don't like you know don't lose superior what's your opinion on that and also of what's your opinion like so many use cases if you can develop on Windows you of your forced to use Linux kernels on your images yes thank but now we continue to fight this battle with Dr. from candles and it's but it's not as if it were something has a mediating that's not working for you know you can just use the multiple ways is shown in this promising and then running the Linux kernel and then is the commission or whatever so it's a connected and what's worked with but it's not working for you you can find a lot because it's still in that many of the guys will help you and because you have got uncover this problem cool as well my so going once going twice of the has