Dockerized pytests

Video thumbnail (Frame 0) Video thumbnail (Frame 7335) Video thumbnail (Frame 16770) Video thumbnail (Frame 26205) Video thumbnail (Frame 27559) Video thumbnail (Frame 30717) Video thumbnail (Frame 32930) Video thumbnail (Frame 33728) Video thumbnail (Frame 35071) Video thumbnail (Frame 35779) Video thumbnail (Frame 38283)
Video in TIB AV-Portal: Dockerized pytests

Formal Metadata

Dockerized pytests
Title of Series
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Dockerized pytests [EuroPython 2017 - Talk - 2017-07-13 - PyCharm Room] [Rimini, Italy] When working with pytest and custom plugins things can get eery when trying to mimic your Jenkins environment for debugging your tests locally, specially across your team and their workspaces. In this talk will go through the challenges faced to containerize a wild pytest environment. Topics: Running Pytest on Jenkins jobs Pytest custom plugins Challenges for locally debugging/replicating a Jenkins job/pytest run Dockerizing pytest and it's dependencie
Web page Computer chess Point (geometry) Intel Software developer View (database) Multiplication sign Demo (music) Computer-generated imagery Disintegration Virtual machine Set (mathematics) Coordinate system Number Medical imaging Coefficient of determination Blog Forest Software Software testing Software framework Statement (computer science) Information security Social class Self-organization Plug-in (computing) Building Coma Berenices Bit Evolute Windows Registry Latent heat Arithmetic mean Process (computing) Pointer (computer programming) Integrated development environment Software testing Parametrische Erregung Library (computing)
Logical constant Complex (psychology) Context awareness Parsing INTEGRAL Multiplication sign Port scanner Set (mathematics) Parameter (computer programming) Function (mathematics) Medical imaging Order (biology) Coefficient of determination Computer configuration Software framework Error message Link (knot theory) Building Structural load Point (geometry) Sampling (statistics) Bit Price index Demoscene Message passing Process (computing) Computer configuration Sample (statistics) Repository (publishing) Configuration space Software testing Data logger Right angle Quicksort Freeware Volume Point (geometry) Slide rule Overhead (computing) Service (economics) Computer file Disintegration Virtual machine Coordinate system Number Root Profil (magazine) Software repository Operating system Software testing Plug-in (computing) Module (mathematics) Plug-in (computing) Forcing (mathematics) Content (media) Code Planning Volume (thermodynamics) Directory service Uniform boundedness principle Word Integrated development environment Personal digital assistant Statement (computer science) Local ring
Process (computing) Virtual machine output Software testing Parameter (computer programming) System call
Scripting language Point (geometry) Computer file Control flow Instance (computer science) Theory Function (mathematics) Network topology Video game console Software testing Software framework Exception handling Chi-squared distribution
Computer file Computer configuration Software testing Form (programming)
Goodness of fit Software testing
Logical constant Root Personal digital assistant Network topology Chemical equation Content (media) Planning Software testing Directory service
Multiplication sign Usability Flux
Point (geometry) Medical imaging Default (computer science) Presentation of a group Process (computing) Profil (magazine) Multiplication sign Computer-generated imagery Execution unit 1 (number) Set (mathematics) Software testing
Complex (psychology) Multiplication Multiplication sign Weight ACID Content (media) Set (mathematics) Cartesian coordinate system Encapsulation (object-oriented programming) System call Wave packet Product (business) Data mining Process (computing) Integrated development environment Visualization (computer graphics) Uniformer Raum String (computer science) Order (biology) Point cloud Software testing Software framework Quicksort
the URIs for coming was expecting this many people and that's the same thing as an inverse means that limit by
introduce myself them enemies 1 some of our forests and I do quality ngineering from our red had in general public for its precisiation solution which is overlaid on the upstream so doing this talk I'll try to go that fast and regulate spin long weeks so uh and to being with unlimited talking a little bit about uh London actually show you a simple test scenario for running fighters on Jenkins and women in heavily view of a high overview on by discussing leans and how pointers that notes because means and then we will believe it but the challenges faced at the time of trying to replicate these uh dangerous job on your local environment we the local and we will look also at the dog rests a solution for for doing this and then if we have enough time I'll make a small like them and if it doesn't fail miserably India of a noun not going to be talking common be known intent to Switzerland so start using containers you should be aware of the security implications that comes with containerized your obligations and 1 of them the Watkins arise in jinkies that's already been done and it's a really if you will already have some some some not highly overview on thinking despite us and Ogre and also there multiple would think that this is about the doctor by library which is for reminding uh Europe containers through by the NGO but that's not the case in in this talk it now so why they decided to to talk about this and without having we came up with with the solution for uh for consideration Jenkins uh now when I 1st set and came to 2 red hat II I was given the task of uh replicating uh might Jenkins job our testing and framework on my local environment and for this so I was even 6 pages document that an it actually took me at approximately 2 weeks to complete and and then as when I realized how big of a fool's errand most to to try and do so a class like I also realize that most of the people my team was uh was all that was having the issues 1 trying to replicate their environments as well but at the same time those environments that were successfully working at some point with the constant evolution of well for testing framework uh boasts setups were easily lost so that's there there had to be a way to to keep up these the people with the same hard centralized and uh make sure that no 1 had to waste 2 weeks of their time to set it up now at the time uh there where we were not using visual environments therefore you had to install uh the uh all the requirements of from the testing framework on your local machine and that could potentially cause some dependencies the some of them is the issues on the libraries that you already had installed on your machine now to also like that the use and benefit of these walls to be able to put uh doctor image on an internal register that you can run on your organization so that 1 or a newcomer uh started working chess by pulling the image would be able to start dividing straight away now how many of you have a been using bytes a stone Jenkins already to it's number so at the this a couple of things that if you need to take into consideration at the time of trying to replicate these jinkies environment uh or this job around uh so 1st thing uh if the job that you're trying to replicate this parametrize you should uh look at the
parameterized options on on Europe on your dog on the duration of your job and you can have all sorts of things and these parameters can be used by Jenkins in many other places you in this that Joe execution uh in in the case that I will show you this sample scenario this is most of its value uh and we only have a parameter that will be used on 1 of the builds textile social you further on no you might also have distributed it tests and you could also have parameters for determining to which hosts you want or on which hosted you want job to run so that's 1 thing to take into consideration then and if the job of it sells itself has any DT integration on the eve the you're actually is is the most common practice so you probably do so and so it's just the it's important to check that all those repositories there are going to be used for on local environments so uh this is this is really important and we will actually link all those repositories 2 volumes on on our container now uh finally a I would look at the preferred the appeal steps on on the on the configuration of the doing his job and there you will depend on the complexity of your existing framework but you might have uh and this case I'm just I should have 1 uh 1 build step where it's it's actually a dynamic much street that takes 1 of of the the parameters of the defined before for choosing which test to run and there you might have additional build steps that could have for example in our case that generates configuration files that are used by custom planes and uh you need to make sure that all those steps are covered before you decide to run by its is on your on your contain now there a by test bitterly all this here all uh I would like this busily any Python module can be reduced us playing so hello bytes is thus there the plane going how how we get the checks all the planes that are testing framework if 1st starts by noting all the built-in planes that were installed with by test secondly it looks for a mam it actually loads all the planes that are adjusted through a set of tools and your point uh I will show you a small sample on the next slide where I am actually passing 1 of the 1 in 1 plane that it's outside or just a framework through uh the this adopt tools a 3rd thing that it does speeds free scans the command line for the minus P name option and it lost those specified before doing the actual command line parsing of lightest uh in this in this case you can also choose to deactivate already registered plug-ins by uh prefixing uh the minus the option of actually I didn't expect this to show later but is minus P Nolde's semicolon and the name of the plane and so that's how you avoid and alone in planes that perhaps issues on your own it's isn't framework and the next thing that the widest notes speeds it looks for all the cost us but by files as inferred by the command line indication and you've you there don't have any test fights they're just past specified uses the current directory as the but now you to uh notes that by test does not find constants the by files and deeper nested subdirectories at tool starts up so it is usually a to keep your constants the py files in the top level of of your of your now next the best uh in that's also load it recursively all the planes specified by the these these fighters underscore plug-ins viable that you can be specified inside your test so that is that actually lost their place where it by just looks for a custom points so now if you if you want to make your planning externally available you may define was the so-called entry point for you're distribution so that just finds you're playing morning and and 2 points are are if you to from feature that it's provided by the by set of tools and like this looks that specifically for the by just 11 I can see no here here here it looks for the center point and as a it actually so that it actually looks for these and point and it loads all the planes that you have defined under these and point uh right so now uh 1 e wefirststarted 1st started to trying to containerized these environment I started by creating a base image with filler and a couple of instructions that I already knew that had to be and for example 1 of the 1st things that I had to do was at the time of linking uh my local
repositories to volumes inside my container and I realize that had to include all these uh all this uh the repositories into might be my Python past so I'd did most that by including the and the instruction on on the the profile for creating the image and later on IEEE started trying to install all the requirements that were defined on on our testing framework uh requirements that txt file inside inside the the root directory of all of the this and framework and it was then when I realized that some of the requirements that I was is trying to sell with people had a had a scare Heather follows from and that were not missed were missing or were not included on uh any place in the on on my testing framework because of all these dependencies where either uh the guy the already including the operating system that was being used for running this test error or being permission by a foreman or other uh and services that wear and diesel to my testing framework um also how are you going monies to uh see we each planes were loaded fighters that every point that was trying to containerized these will use in the trace config option from by test we actually shows you all the planes that are being recognized by Pub quite as of the time and now as you can see that I have these uh run statement that I include a my doctor file no and it is recommended that you minimize the number of of commands goes therefore that in that way you you it makes for less overhead at the time of actually running your contain the so now when when you're linking in volume on your container to a local rebel directory on your on your machine if you have a B S C leanings policy in force then uh the container won't be able to write to these directory and in the case that you you might be possible that Europe Europe your tests are generating log files that save on that same a repository and if you have a xylenes enforce the container will fail with permission nite or under IBCs a message in the host sees look so all to avoid that and you need to change the context of the directory of the of the in this case the repository that at t to the to that time in particular as read some books filed the now this has been the this has become much more easier since Dr. 1 bonus . 7 because as you can see on the on the 2nd 1 the 2nd letter uh piece of gold here as I am pretending the words that I am prepared and the set at the end and scenes Boca 1 . 7 these uh the statement automatically changes in the context of the directory that you're trying to link to your content and therefore the content will be able to write to that reported now and have further on at the very end of my doctor file II I defined these 2 instructions the entry point and assume the now the entry point to eat Beardsley makes your container executable how soul because when you run your container image uh the 1st command that the container will run will be the entry point and this year the it's any initial options that you would like to run with your point but wasted separated like this it's because they're the CMD eats over rights of like you can overwrite that by adding options to your run statement for 1 4 when you want to run your contain and therefore it makes it a more and more flexible now in at the time of the body in Europe Europe uh you might want to amateur I use IPD for example but and use PDB or pi exists on PDB uh do you won't be able to to the but because by this is capturing the standard output so into you need to run bytes this with the minus as options so that you're able to use any of the Warriors apart from the or as the so the 1 for the real fast now I will show you a little bit fall them on is small and the small scenario their marriage and things uh you can find a and that the testing framework testing framework that we're going be using on these let them all on these repository and there's 2nd repository contains the Booker files and the doctor composed file that I am using for associating these Indians environments so I and move we'll see it but of
the call so this is the diseases that have running on a montane cost my local machine and these Joe in particular there is a parameterized job that takes uh to takes only 1 parameter and is based on the input that we put them people will run 1 test or the other as we saw on the on West Street before so all run about test and a good test 1 should fail the other 1 showing uh most likely both of uh of so we got a failing test here and no 1 successful the fate I want to I want to divide this failing test that it's running on minding
seasons now I already
be a really created a container in each of their that will replicate my you might to
induce instance and a container I can use for that the might might bite is locally known have some scripts here the so if I run that differ on this this script for running the SVM but can show you how old what does the following things and is basically a running instruction for for local to run out with the FC 24 container that I created uh with with the delta file and basically I am telling straight away to buy tests which test to run so now in theory if I and this is the testing framework so in theory if I eat it needs my let us to include the these breakpoint then I should be able now to run my container and I we should be able to achieve that break on and yet they see so time can't and because that actually a dozen it actually uh well yeah you know the the it it it's probably it's the actually hits the break point where when and an exception is raised it also did actually to stop the execution when our uh and the trees what is right now you might have no for non freely tests that you will still want about so then good soul
let me show you I get our streets here as well uh let me show you I have defined on my cost they have defined as small playing a small custom claimed that it is in the form of a lot of Mark and I am using that all might be good for us not so so basically is marked these 4 including an option name on the bikes it's run that will look for any test that our attack whatever I introduced on on my and I only by is right so here no we yeah so if around the the good file now the these uh and this file in fact MIT has as you can see I am passing the minus the options and their the good tab the was fine on my bed on my marker for the
test in particular so the if
I tried to run this container yeah I change the and change the label to whatever it will actually skip the test
because it is not the label for good that we had defined on on the test now there additionally we can
see with the trace coughing deceased also
just running the fighters with the trace conflict and on the trees you will see all the planes that have been reduced by uh but by by test as well as the name as well as the cost tests that have been loaded by by so if I do at this you can see see like in this case it's showing me that it has loaded the constants from the root directory which he's on the container itself cause I am I am rooting my willingness and linking my balance to the main road folder on on on my content so pretty much it them out yeah so
all yeah that's that's pretty much it now if you if you have any questions of free will not have any in more time going when it is true that look
at so you so high f
for for these containers by the glass I'm going to yeah sure this so
these these there uh the 1 for the Jenkins uh he says that it's actually begin up the base image from the over harvesting is latest an idea include some some other instructions for a pool in unit time uh then you get the think slave butterfly which is just the job of governed in this container and there I am downloading the the the divergent from doing things for a executing the the following so you can find these all you have a Yi there you this you can find the the profiles there and so was the entry points and stuff where are they I voted for this for this study in particular I can use any entry point or in the but it's like it's just like you can only use CMD ones and the entry point should be your your default command to the executed with the and thank you for the presentation was very nice set up so again and again thank you the presentation i was a nice set
up I have 2 questions from 1st of all this looks like it I want the benefits of this might be able to run multiple tests from Jenkins at the same time is is something that you've tried that it works properly with the set of yeah you can you can run them and then it test that you know 1 of the the framework if you already have set up like that on your drink uses Justin concessions was just curing them because the charge against the string is is as was just there to to show that 2 different places were like the Jenkins job that is running the by as on my local quite is run that will be able to be but notice a clear call ominous question do you can you can you tell me what that show you had with it should whether the kid was dirty your claim that the train I select this is all mine that set S with some takes the occured thank you want and I know I think a products very that about on 1 let my question is a that is when it comes up in my understanding get personally when it comes to doing your application I can understand the benefits of using the AUC about another that then this is like an encapsulating ever that all of the been this is inside your contain have so you can should because is order uh make it a medic up making obligation that cloud native reading some supplies of stuff like this but talking about distingue only what is that the advantage of using the content I solution with respect to other available solutions that in my opinion might be more light weight solutions like visual environment so SIV consider the set of the solution and that understand what is your opinion again because as I said before give you given this complexity of our testing framework uh these wars the best solution at the time and uh I mild acid said also and I'm not going to and I want preach short-sea Tokyo into use in Boca or containers for for replicating agendas seasons but uh this this is actually saving us a little time for newcomers and also uh it's maintaining some sort of uniformity on on on our on our test in France do we have any more questions known and thank you very much the value more