Add to Watchlist

The realities of open source testing: lessons learned from “Adopt pytest month”


Citation of segment
Embed Code
Purchasing a DVD Cite video

Formal Metadata

Title The realities of open source testing: lessons learned from “Adopt pytest month”
Title of Series EuroPython 2015
Part Number 49
Number of Parts 173
Author Laugher, Brianna
License CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
DOI 10.5446/20092
Publisher EuroPython
Release Date 2015
Language English
Production Place Bilbao, Euskadi, Spain

Content Metadata

Subject Area Computer Science
Abstract Brianna Laugher - The realities of open source testing: lessons learned from “Adopt pytest month” Ever feel like your open source project could be better tested? Lack of tests holding you back from contributors but you don’t know where to start? You’re not alone. [“Adopt pytest month”] was held in April 2015. [Pytest] volunteers were paired with open source software projects, to find a path to better testing with pytest. Projects varied from libraries/command line utilities, to a browser, to a complex Django app. In some cases converting existing tests was necessary, in others writing the first tests in existence for non-trivial amounts of code. Two projects were open sourced specifically to take part in “adopt pytest month”. What began as an experiment in increasing software audience proved to be an interesting exercise in strengthening community and most valuable of all, provided a newcomer’s perspective to veteran contributors. This talk will discuss what worked well with “adopt pytest month”, what didn’t, what we learned about pytest and what you could take away for your open source project, be it an improved testing environment or an improved contributor community. A basic knowledge of testing and pytest will be useful.
Keywords EuroPython Conference
EP 2015
EuroPython 2015
hi everyone and welcome thank you for coming and now before I get started the it was actually involved with the doctor quite as
much semester my but not quite as masters of India and this is a contributor to an open source project 1 right so nearly everyone great so it probably
1 surprise you to know that I love 10 of being applied the user a few years of contributed a few patches and I generally find the eternal
joy to you and it was folded founded by hold the critical book you gave this morning's keynote has been around since 2009 it's a very mature stable projects have full of features and has around 170 plugging to natural quantitative framework and so in January at the 1st and Conference in Brussels we had a little practice made up and it was lots of fun we got to talk about all kinds of things we're excited about doing all fixing Pietist and I thought to myself why don't let
everyone use part 2 but some libraries for example request everyone everyone uses requests and around the and just use request but that's not really the case with protease and has many many friends and many people strongly recommend it but it's not really in situation that people say just use end of story I mean we even had a meeting with the you know sticks with that unit test so why doesn't everyone well there's a
few reasons that I can think of and some of them are out of control we will never be able to satisfy the but there are some that the putschists community the the platters contributors could do something about to encourage more people to use so that was then
I had this idea that we
could carry out open source uh maintainance who were interested in getting started with life we've experienced test users or developers and they can work together for a limited period so as not to much of a commitment had to help them really get started and get off on the right foot with practice and that would help overcome knowing where died knowing if you're doing and really taking advantage of all the the best features of context and with anyone at the end of the month we'd have a host of new practices which would be great I decided to organize it for a group and so I read up on a page on the party's website describing what will planning and I mentioned surveys to sign up the parties and help this and also the open-source maintainers and 1st I tried to find out the Pitons tokens because I thought well if we are inundated with open source projects we want to make sure that we have for everybody and so I asked the tightest help was the experience they didn't have to be applied to the developer so they did not have to have contributed code to plant Test but they have to feel that they were an experienced in New year most of the most important features of practice and ask them about their experience with other areas of Python programming like and frameworks like Django or scientific programming so that I could try to pick people with appropriate
interests and experience and we had 26 plexus help find out which
there was a lot more than I was actually expecting you saw the practice been out there was uh 6 of best thing that them and so I was really surprised that 26 people want you to help other people when practiced and only 9 of those people were people who had actually contributed to contributed code to practice before so found 15 new people who were as a keen to advocate for practice and then next I tried to sign up to the open source projects I wanted to reach open source projects that were fairly well established and so initially we had a requirement that they should have more than 1 call contributor and after a couple of weeks we do not have many sign up and so the case maybe we can relax this requirement and uh so projects with just 1 contributor that's fine as well and the main way that projects actually found out about the departures months and to in used by being approached by part of the users so we asked the parties community to look at other projects that they were part of more aware of and maybe ask them on their have issues on their mailing list if they would be interested in taking part in the top right now and that was by far the
most effective way to find people who were interested to take part and in the end we had
a projects that kind of officially took part in adopt quite as much and there was another for that kind of informally took part in that they said yeah sure if you send me a pull request all accepted but they were not really committed to kind of working closely with their practice help and so the products we
had were uh bodies which is a library which has provides a bi-directional dictionaries guessing is a command-line utility to extract information from media file name calcium is a source code management systems similar to get lab which supports both Git and Mercurial keep browser is a keyboard-driven browser based on qt find that the control in a similar way to trump is a framework to 1 pandans that aims to centralized management out of the Arkestra is an extension to gender that provide Semantic Web publishing and is used by a bunch of organizations including the recent Django corn in the case Net retiree is a REST API framework sitting on top of the pyramid and elastic search and Coursera DL is a command-line utility for generating cost there at all media and champ and if retiree would both open source by by different companies specifically in order to take advantage of adopt artists small so that companies that were thinking about open sourcing the predictable planning to eventually do that but I don't want to was really the the trigger for them to actually do it and informally had Jason people cookie-cutter ginger to and pelican and the last 3 actually accepted pull requests before April even began so they kind of as why I said it informally took part as they did benefit and practiced did benefit from and contacting them but they did not really work through them up and so
given that every project would be in a different place with regards to the testing I lifted up to the parties told and maintainers themselves to decide and what activities would be appropriate what would they how much would they tried it on during the month but I just suggested to working on incremental steps that built on each other rather than trying to change everything in 1 big bank and to do lots and lots of card reviews like very intensive code reviews and to spend a lot of time just talking about the the tightest features and you kind of wondering out loud is this possible can you do this and having an opportunity to to learn through discussion and so after the month was over I centers so they have to all the participants to ask the and what they didn't have
so the the point is that I hope this did things of course like updating tests changing unit test set equal statements to claims that statement and changing the layout of code test and dogs to well I'm not sure that was full practice so it's just kind of incidental cleanup and removing boilerplate plate code that he needed anymore fixing minor bugs that they actually discovered through this process converting tent tests to use parameterized and making custom fixtures and custom markers and introducing the use of various heightened plug-ins and
so on self-reported I ask them how much time did you spend on adopt quite as much initially I advise maybe 2 to 4 hours a week was the time commitment not really knowing if that was a lot of time or not very much time and really varies I'm not really sure how accurate these numbers are because estimating you're
remembering back how much time you think can be quite difficult that's a 5 of these protestants said they would definitely take part in it again if I said they might take part in 1 % that they probably would not and I ask them how do you feel about the practice community now after the months and 5 said they were more positive and came to be involved in the community and 5 said they'll have a similar level of commitment which might already be quite high in fairness and they also reported learning about different features of that they hadn't been exposed to before and from various plugging as to how the run test how to you support artists in various continuous continuous integration server and more about tightest support for unit test and noise tests weight then I ask the
maintainers to tell me what they did and of course they as reported discussing the plans over e-mail or chat and no 1 in projects used a chapter which is built on top of it how others used e-mail or just the issue request issue tracker in it have itself and they write new tests and put pull requests for the participants to give them feedback and 1 enterprising maintain actually updated apart as pointing to work with to divide and 1 project was Sergio made Popeye released for the 1st time so it's a very coarse idea is actually very popular projects by the number of users is not proportional to the number of contributors which is not uncommon and the maintainers sedimentation 80 hours of what's really going above and beyond and at that time varied a lot so far I'm maintains respondent others did not respond to my 7 end for them said they would definitely well they would recommend taking part in adopt quite as much to other open-source projects ones that may be and they recorded that how they felt about point is now that they have a good grasp of the basics but there were also aware that there was really a lot more so that they could learn and then they
felt very positive about actually and they all indicated that they going to
keep using practiced in their projects and 2 of them told me that they have actually introduce pike as in other projects that they contribute to which I think is really cool so even though we directly on worked with a projects because many open-source contributors working multiple projects we have a multiplier effect and in general people who responded to me told me that they really enjoyed it and I really grateful for the opportunity and so maybe some who didn't apply didn't have such a good time at which is no surprise that in FIL levels um but just
based on my observations of looking at the activity in the various projects uh Of course there were not all runaway successes so this is my observations about some of the areas where people struggle and they're
not really surprising that the kind of problems that we face in open source in general over time so you get excited about something you think you're going to devote a lot of time to it but relax intervenes and you end up not being able to do anything and so they some parties helpless really show up and I mean I knew nothing about them means that they have completed the survey for me and some of them showed up but we're not really able to make much progress in the product and some of the maintainers were extremely busy and not really able to work with the help of the
size I wanted to think about what are some things that we've learned from this that if we do it again we could take into account to try
to have a better experience for everyone but because even the I told everyone you know it's all volunteers and we hope that it works out on people were very understanding but at the same time it's kind of disappointing if you think you're going to work with somebody but then they don't actually show up or you don't get done what you wanted to get started so it happened we have realistic expectations about what we're doing so the 1st thing I learned is that you don't really know how many people are looking in your community so for those of us who are experienced in
so we kind of knowledge that the way to get into it is to just march writing see right on and start working on something and submit a pull request and that's fine that's actually the way it works but when you are outside the circle is not very clear how do you how do you didn't how do you stop and it's kind of intimidating so giving it a foothold to people say this is the thing that you could do it is like a very clear expectation about how you take part in this activity and was a really helpful thing to do and 1 of
the parties help said these they said that it was the 1st time that contributed code to someone else's open source projects so I think that's typical
we were able to bring someone inside the circle and to me this was a and a really important revelation of doing adopt as months that uh there's a lot of value in making a space for users to be advocates of your software so I think it's I mean people who are excited about bringing other people on board to projects that have to be contributing cause uh in companies they often have positions called evangelists and developer Evangelist so common you know give talks at conferences and tell you how great library is that you should totally use and uh in open source projects that are not corporate tax or company that's that's usually left to the developers but there's real valued in recognizing the work of advocates in our open-source projects it's not only good for the community but is good for the users of your software and so
the sensing is that can't be is maybe not just important actually amazing so I I called my talk the realities of open source testing and this is actually what I'm talking about because
the majority of open source projects probably the median number of contributors is what so it a real long tail distribution of open source projects and the head but really popular projects like Python gender even contest is fairly
far the the head of the the graphs and but there is just this long tail of hundreds of projects that have 1 contributor and even a popular project like Coursera DL I mentioned only has 1 person is really developing code and so for them is incredibly valuable to have someone reading the code engaging with it and able to give them a specialized advice about the way that using a particular library and a testing library is something that you really rely heavily on a new test could might be 40 per cent of your whole code that's so you really wanna make sure that you getting the most out of I mean the the parameterized mockery
in part this is a good example of that you can easily right very repetitive tests without realizing that even the but if someone tells you hate this exists you can condense and 500 lines into 50 lines that it would just blow your way and so the last thing is that as something I
really underestimated is that this kind of projects I was thinking a lot about how we add teaching moment or in the source maintains about hottest but it was really also a case that for each helper they were they would need to be made toward on border onto specific partner projection
and I didn't really emphasize this to the project
maintainers and of course it in hindsight I can say of course it makes sense that you need to understand the domain of this piece of software the concepts and the functionality most of all before we can effectively tested and you need to have their documentation in place to support all and which is more than just having docstring so this is something that would be aware of the 1st to do this again and so my question to
you to the open source maintain is in the range is can you make a space advocates in your project always do you recognize the work that advocates of doing in your project and could you offer a kind of targeted code review program to to users of library and if you were interested in doing something like that uses a few suggestions and something important is that you don't necessarily need to do this we new users as we did with the doctor part of not it would actually work probably even better with existing users just helping them really make the most of using the library and I put here a encourage people to say sorry I haven't done that yet or I'm not sure where to start and I think a it's very common that when we are blocked and or if we procrastinate and we don't something that we told someone we would do and they write an e-mail and they're like Haiti has that thing coming along it's kind of it's very hard to reply and say I haven't done that
yet is it's much easier to just kind of ignore the e-mail and then as the week goes by and the real so a lot of time in an initial period of the projects like this which only runs for 1 month so making it much trying to tell people is really OK to save either I haven't done that yet or I don't know where start and would help to overcome some of those communication barriers or uneven contributions and I think it's really important to recognize you advocates and try to recognize people who doing work in your projects that are not that's not contributing
card so as something important for me it was made the point of right printing and recommendations for the advocates as a way of and acknowledging that they dont have really good really valuable and really important work for quite and definitely don't schedule in the same month as pipeline 1 US which is what I did take notice so to conclude uh it was a very interesting experiment and I I think it's been really exciting to meet some of the people who took part in it I have here I Europe friends and I 1 of them predicts maintainers actually delivered the practice training that we had on Monday so in this space so that less than 2 months these gone from not knowing or not really knowing Titusville to delivering training on it and updating plug ins and all kinds of things so as I think it was really valuable and interesting a positive experience for contrast and I'd encourage other projects to think about if it is something that they could do in some way and said thank you very much to the the departures contributors to kind
of supported trying this in the 1st place as well as the the helpers in the project maintainers patients as will kind of trying thing and not really sure how it's going to work and I just wanted to
mention that tomorrow uh there to talks from the to practice helpless and talking about slightly related things and more information including a PDF report about the hottest month in case you wanna get all the details about it happens addresses so thank you
few new dimension to the woman to questions well 1st of all thanks a lot I 1 of those guys do use that as an adult contributes theory but again thanks well that's just as part of the question is now how do you feel about effort that could be spent all of that the product especially that libraries through social media for example if I search Twitterfall quite as I got that Oracle was others many all over so and so on so this that is important to maintain such a channels to interface because 1 or developers sporocarp happens just enough it's i haven't thought about it it's important in general but I started the part startled Twitter account about the time of that we started thinking about adopt widest month ends a tried to use that to advertise the concept on Twitter which work the case but actually something really great about their Twitter account is that many people just post tweets saying about how much they love scientists or they're like giving a talk about how it is that the user groups that we had no idea about like this talks being given in Portuguese Japanese time of all around the world and so uh just having an like search set up for any mention of has revealed all these things to me and which is really really it's so nice to see that people love something that you apart and we had a meeting yesterday were actually talking about setting up a blog and to does that kind of information as well so I don't know if I I would say it's essential but I I think it's and a positive thing and I think having a blog where we can post a link to tutorials and things will also held a users to see that it's so there's a vibrant community around practice and I think that's something that people take into account when evaluating whether to use a library on it's good to go to show that you have a lot of users you have people working to help you if you need help so that it wasn't on the tangent plane in which
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation
Computer animation


  397 ms - page object


AV-Portal 3.8.0 (dec2fe8b0ce2e718d55d6f23ab68f0b2424a1f3f)