We have nearly one million lines of Python 2 code in production – and now?
This is a modal window.
The media could not be loaded, either because the server or network failed or because the format is not supported.
Formal Metadata
| Title |
| |
| Subtitle |
| |
| Title of Series | ||
| Number of Parts | 130 | |
| Author | ||
| License | CC Attribution - NonCommercial - ShareAlike 3.0 Unported: You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this | |
| Identifiers | 10.5446/49937 (DOI) | |
| Publisher | ||
| Release Date | ||
| Language |
Content Metadata
| Subject Area | ||
| Genre | ||
| Abstract |
|
EuroPython 202066 / 130
2
4
7
8
13
16
21
23
25
26
27
30
33
36
39
46
50
53
54
56
60
61
62
65
68
73
82
85
86
95
100
101
102
106
108
109
113
118
119
120
125
00:00
CodeLine (geometry)Software developerArchaeological field surveyVideo gameGroup actionPerfect groupLine (geometry)Archaeological field surveyCodeHuman migrationSoftware developerSoftwarePresentation of a groupResultantDisk read-and-write headProduct (business)BitMultiplication signVideo gameZoom lensDigital photographyComputer animationMeeting/Interview
03:22
Bit rateArchaeological field surveyVideo gameHuman migrationCASE <Informatik>Level (video gaming)Enterprise resource planningService (economics)Cartesian coordinate systemFormal languageSoftware testingProduct (business)Revision controlMereologySoftware maintenanceProof theoryDifferent (Kate Ryan album)Multiplication signSynchronizationProjektiver ModulDampingException handlingStress (mechanics)ResultantBoss CorporationInternet service providerSoftwareRule of inferenceSoftware developer1 (number)ThumbnailProcess (computing)Human migrationContent management systemBacktrackingServer (computing)Presentation of a groupFitness functionEvent horizonSoftware bugMathematicsOnline helpDatabaseInstance (computer science)Scheduling (computing)Video gameArchaeological field surveyCodeTask (computing)Content (media)Canadian Mathematical SocietyPhysical systemRewritingComputer animation
11:59
Human migrationData typeCodeSoftware testingMereologyElectronic mailing listRevision controlSoftware testingArchaeological field surveyRevision controlDatabaseCodeHuman migrationContent management systemCore dumpType theorySymbol tableFunctional (mathematics)Presentation of a groupSocial classConfidence intervalWeb 2.0Software maintenanceCartesian coordinate systemElectronic mailing listLine (geometry)Content (media)Text editorMultiplication signInformation securityHazard (2005 film)Formal languageBuildingDeutscher FilmpreisStack (abstract data type)Slide ruleSource codeModule (mathematics)1 (number)Subject indexingState of matterObject (grammar)MathematicsMehrplatzsystemNumberMaxima and minimaMereologyGroup actionImage registrationDirection (geometry)Variable (mathematics)outputExpected valueScripting languageFreezingFunction (mathematics)Wave packetCanadian Mathematical SocietyDifferent (Kate Ryan album)System callOnline helpPhysical systemData managementComputer fileComputer animation
20:36
Electronic mailing listRevision controlHuman migrationNetwork topologyCodeSoftware testingDatabaseInstance (computer science)Moving averageMultiplication signLink (knot theory)Sheaf (mathematics)Web pageLatent heatLibrary (computing)Axiom of choiceNetwork topologyExterior algebraWeb 2.0MathematicsMereology1 (number)Revision controlModule (mathematics)Instance (computer science)Software developerHuman migrationCodeRepository (publishing)ChainOcean currentSoftware testingPresentation of a groupPlug-in (computing)Content management systemDatabaseGraph (mathematics)Installation artDisk read-and-write headCore dumpCartesian coordinate systemCASE <Informatik>Product (business)UnicodeString (computer science)Error messageLevel (video gaming)Software bugObject (grammar)Search engine (computing)Relational databaseProcess (computing)Text editorIntegrated development environmentUsabilitySocial classAsynchronous Transfer ModeBranch (computer science)Single-precision floating-point formatHash functionMultiplicationEqualiser (mathematics)Electronic mailing listPairwise comparisonData storage deviceCanadian Mathematical SocietyComputer animation
29:13
Human migrationDatabaseInstance (computer science)Software testingDrop (liquid)Module (mathematics)Broadcast programmingMetric systemRevision controlMathematicsRevision controlRoutingBranch (computer science)Module (mathematics)DatabaseError messageServer (computing)Software testingFraction (mathematics)State of matterMultiplication signContent management systemSoftware developerStreaming mediaLatent heatIterationRun time (program lifecycle phase)BitMetric systemScheduling (computing)Open setPoint (geometry)Software bugData conversionHuman migrationPhase transitionIntegrated development environmentLevel (video gaming)Drop (liquid)Single-precision floating-point formatUnit testingCartesian coordinate systemDifferenz <Mathematik>ImplementationString (computer science)Library (computing)MereologyWordControl flowTask (computing)Product (business)CodeProcess (computing)CASE <Informatik>Installation artExecution unitPresentation of a groupNumberCanadian Mathematical SocietyReading (process)Computer animation
37:50
Software testingPressureCodeSoftware developerSoftware bugMereologyHuman migrationCartesian coordinate systemRevision controlService (economics)Process (computing)Data managementChemical equationOcean currentIntegrated development environmentRight angleSystems engineeringArithmetic progressionValidity (statistics)Perfect groupInformation securityError messageBranch (computer science)Computer fileResultantDifferent (Kate Ryan album)Complete metric spaceMultiplication signInstallation artWritingException handlingCanadian Mathematical SocietyTask (computing)Computer animationMeeting/Interview
Transcript: English(auto-generated)
00:07
Perfect. So you're gonna talk to us today about the transition from Python 2 to Python 3, right? Yes. Okay. Perfect. And are you ready to start then?
00:23
I'm ready. Wonderful. So Michael Howitz, the floor is yours. Hello, everyone. Welcome to my presentation. We have nearly one million lines of Python 2 code in production and now. In summer 2016, we started to think about this question.
00:43
I'm going to present how we solved this problem and what you can learn for your own big and small Python 2 migration projects. Let me start by introducing myself. Oh, I see. This is a bit older photo. Well, since 2003, I am a Python software developer. I was the head of development
01:07
in the Python migration project my presentation is based on. My employer is a small software and consulting company named Gosept. It is located in Halle, Germany. We do not have our own
01:22
products we sell, but we develop and maintain projects for our customers. Currently, we work on some migration projects, migrating them to Python 3, or we even have finished the migration. I'd like to say thank you to Gosept for the time to prepare and to give this talk.
01:49
Now you know a little bit about me, let me ask a single question to you. I took this question from the Python 2 end-of-life survey done by ActiveState in the last quarter of
02:02
2019. The question might sound a bit off now, but nevertheless the question is how prepared do you feel for Python 2 end-of-life? There should be a button to answer this
02:22
survey for you in the Zoom, so let's try it. I can answer the question myself, that's nice.
02:45
Yeah, we have 48 percent of the attendees voted so far. 50 percent. Come on, bibs.
03:01
It's 28 of you that haven't voted yet. 27. Okay, so I'll try to present the results of the survey and then we can compare with the results of the poll. The results of the survey were
03:26
that about half of the possessive pants felt highly prepared, but about a third did not feel prepared at all. So I do not see the results of the poll.
03:42
No, but so far we have 27 percent not prepared, 45 percent somewhat prepared, and 27 percent highly prepared. Okay, that's even less than the survey results, at least the ones who are highly prepared. So the end-of-life of Python 2 is already history.
04:04
If you do not feel well enough prepared to handle that event, let's see what this presentation can give you. I schedule the following four items for you. Discussion of approaches, introduction into Union CMS as I use it as example project for the presentation, doing
04:24
migration and tips and tricks. Let's dive into a discussion about possible approaches. What to do with an application running on Python 2. Discussion here means that I am the only one who discusses, sorry. I'm going to present the following five approaches.
04:44
Sunset's application, replace by something off the shelf, don't migrate, start over or fade out, and move to Python 3. Let's start with the first approach. Python 2 is after its end-of-life, maybe your presentation too. Ask yourself how important is the business case the application supports?
05:07
How long will I or my company earn money using this business case? Can a migration be a success even financially? This approach might be the cheapest and the most safe one,
05:21
if it is possible at all. But you should at least think about it. Don't burn money by touching an application which is not or no longer worth investing into it. Is your application so special that it cannot be replaced? Maybe you can even replace it by
05:44
something off the shelf. It is probably cheaper to buy a service from a company than doing everything yourself. Yes, I know there was a time when it was totally cool to write your own tools. Nowadays there are lots of specialized service providers who can do a better
06:02
job. If you want to keep your tool, it's your baby and you have to take care of it. On the other hand, you mostly get only an 80% solution. Maybe you can work around the missing 20%. One of our customers, for instance, is replacing big parts of their homegrown
06:22
ERP system by products backed by companies. The bug tracker, the internal CMS, the calendar, and so on. You still have to migrate the data. I know this can cause big trouble but you only have to do it once. After the migration of the data, the problem of the service provider,
06:46
the task of the service provider to keep the service running, to fulfill feature requests, they have to take the maintenance burden. Users might need to learn a new tool. They might require some customizations but being forced to tell the user the new application
07:05
cannot do this, let's find a way around it, might cause less stress than having to ask, when do you need this new feature? Even my company, the company I work for, as a software shop, has a slogan which roughly translates into English, better buy it instead of develop it on your own.
07:26
The next approach might be dangerous, don't migrate. My boss won't like that I mention this approach here because it could ruin our business or save it if you are forced to change your mind
07:41
in two years. If the use case of your application will go away soon, neither maintenance, nor development of your features is required and if there are no bugs to fix, this approach might be a fit for you. But be careful, don't use this approach on critical infrastructure
08:02
which ran for years and nobody wants to take care of. What will happen if the server running the application dies? Okay, it won't be a problem, it runs in a virtual machine or in a Docker container, but does it really? Will you be able to find help if you have problems
08:22
with the code within the next two years? So be aware it will become harder to keep an old application running and to find someone who wants to invest time into it, even if you have the money. If it is all ruins, maybe start over. There are
08:48
directly in Python 3 or even in another language, try to keep at least the high level tests so you do not have everything from scratch. There are many people who say don't do this and I am one
09:02
of them. It might be interesting to start on new ground. In the beginning it looks like a great opportunity to do everything correct this time, like you read it in the book. Ofna, this will become a great nightmare, at least for big enough projects. You can do this for a microservice,
09:21
but not for a monolith application. There might be exceptions to this rule of thumb, but why do you think you are the exception? Rewrite projects are the ones which rarely succeed. Only rewriting is never enough. The new application should also be a lot more flexible
09:42
and have a ton of new features. And you still have to migrate the data. Or do you dare to use the same old database? During the rewrite, the old code still has to be maintained. Change requests now have to be implemented in both variants, the old one and the new one.
10:06
You have to learn the use cases which led to the former application again. Or do you have up-to-date requirements documentation? Users will not be happy with the new application. It's always the kind because it behaves differently. The old one they used to complain
10:25
about will over time become the secret weapon to get things done. The minimal viable product for the new application will be quite big, especially as there is an old working one
10:42
the users know and trust. There is a sub approach to start over named FadeOut. Replace the parts of the old monolith one by one. This might work. But wouldn't it be easier if the monolith already runs on a recent Python version, so you do not have to sink in different
11:04
versions if you have to work both on the monolith and on its successor? If nothing helps, you could even try to move your code and your tests to Python 3. For me,
11:21
this is the most interesting approach because I love legacy code. I admire the people who created it and if I can make it even better, it would be a great opportunity. Of all the approaches I
11:40
showed you, this one I think is the most future proof. It uses what's already running in production and only transforms it. It can be challenging but until now it was possible for each project we touched as a company. Let me introduce UnionCMS. UnionCMS is a content
12:05
management system once written for Verdi, a German trade union. Its first version went into production as early as 2003. It's now also used by degbe, also known as Deutsche Gewerk-Traspoint.
12:21
UnionCMS is a multi-site, multi-user content management system. It has thousands of content editors and it serves its content to some million visitors. Sorry, the source code is not publicly available. When we first touched the source code in 2009, it was based on ZOB2
12:44
and the ZOB object database, ZODB. It was probably running Python 2.6 as Python 2.7 was not yet finally released those days. Over the years, we developed many new features for the CMS, so finally we ended up having nearly one million lines of Python code together in the core of the
13:06
CMS and in the project using the CMS which are built on top of the core. The customers are interested to keep the CMS running and maintainable for the next coming years, so
13:21
there was no way to censor the application. Do not migrate was also a no-go because of security considerations. Replacing it or starting over was no opportunity. The customers invested much time and effort in the previous years to train the editors and to create or migrate
13:43
the content from an older UnionCMS version, so we were happy to migrate the core and the projects built upon UnionCMS to Python 3. Now let's take a look at the migration steps which worked well for UnionCMS. I believe they can be used in Python 3 migration projects of
14:07
any size. I'm going to present five main steps. Step one of the migration is general preparation. This means have someone who knows Python 2 and Python 3 and the differences in
14:26
between. There are enough tutorials on the web so I'm not going to cover this here. It deeply helps to have a decent test coverage. Yes, this means you have to part the tests too,
14:51
but the test coverage gives you confidence that your code still runs. In UnionCMS, we currently have about 88% test coverage of the Python code. When we started, UnionCMS had
15:07
less test coverage. It increased during the migration project. Sorry, I do not have the actual numbers from where we started. Currently, the 88% are quite good enough.
15:25
Annotating each function with the expected types for the input and the output, aka type annotations, might help. Checkers like mypy are able to find problems by statically analyzing your code. But in Python 2, type annotations have to be
15:43
common instead of being part of the language. So in UnionCMS, we decided against using type annotations for the migrationer. It seemed too much hassle for an unknown gain. Step two of the migrationer. Clean up code and tests. This means all tests should be
16:05
running and passing. Broken or disabled tests are generally no good idea. You should try to fix them or delete them if nothing helps. Some parts of the code are probably unused. You should not port them. The ones used will be difficult enough to port,
16:25
so there's no need to work on that code. In UnionCMS, it required a lot of detective work to find code which is no longer needed. There were Python modules which were not even imported,
16:42
so they had no pyc files. There were classes and functions which were not used anywhere. We had the advantage to know the codebase relatively well to find that dead code, or at least hopefully most of it. There was a presentation on this
17:02
conference about the joy of deleting code, I think it was called. In this talk, you can find ideas how to find and delete this dead code. Removing dead code should be done like surgeoner, symbol by symbol, function by function, class by class. Remove everything the dead code needs,
17:26
the imports, the global variables, the registration, etc. Each deleted line does not need to be ported and supported later on. Step three of the migration is bring dependencies to Python 3.
17:41
That means you need a list of all the direct and transitive Python package dependencies of your application. Transitive dependencies are the dependencies of the dependencies, and the getthis list depends on the tools you use. Only two examples, if you're using pip to install your
18:05
dependencies, call pip freeze to get this list. Union CMS uses CC build-out. It lists all the needed dependencies into generate-run scripts, so we got them from there. Each dependency has to be
18:21
checked for Python 3 compatibility individually. This means take a look at the Python package index, aka PyPI, for the package and see if there is a version which declares Python 3 support. I know this step could be automated, but looking at the package gives you a feeling for the package.
18:43
When was the latest release? Does the package still seem to be actively developed and maintained? Which version is the last one which supports both Python 2 and Python 3? On the next slide, I'm going to suggest to port to Python 3 by keeping Python 2 compatibility, at least for a while,
19:02
so you need to know this version. This step could also be used to introspect the stack you're building upon. Possibly you will find packages which should be replaced or have a maintained successor package. At the end of the step, you have a snapshot of the versions you're
19:24
using, the package versions needed for Python 3 compatibility, and a maximum package version numbers to keep for Python 2 compatibility. When you know the needed versions, you can use them.
19:42
Make sure you run on the newest versions which are Python 2 compatible. This might require some changes in your code, so it works with these newer versions. This, of course, also means run on the latest version of Python 2.7. I think it's Python 2.7.18,
20:03
which was released this year. There might be dependencies which are not yet ported to Python 3. You could try to replace them with other packages. According to the already cited
20:21
active state survey, finding replacement packages, more than half of the participants of the survey expected this to be a challenge in the migration. Sometimes you are the chosen one to port a package to Python 3. You could see that as an exercise for the migration
20:40
of your application. Maybe it's even enough to port the parts of the package you actually use. To get a clear plan where to start porting your application, you might need a dependency tree of the packages belonging to the application. It should at least exist in your
21:03
head. The packages without the dependencies, the packages with circular dependencies, which need to be ported together, at least the interdependent parts, and the packages which need the whole core to be ported before. In union CMS, we knew the dependencies well enough, so there was no need for an explicit dependency graph.
21:27
Unfortunately, I cannot suggest a good tool for creating that dependency tree. There might be one which I used more than once. It's called tlactapps, but it might not fit your use case.
21:45
Let's move to step four, migrate the code. There are several tools to help you. Modernize is a packet created by Armin Röhneher, the developer of Flask and many other Python packages. It is used to convert Python 2 code so it can run both on Python 2 and on Python 3.
22:09
To achieve this goal, it adds a dependency to a library named 6, as 2 times 3 is 6. Depending on this 6 library makes it pretty easy to drop Python 2 support later on,
22:22
as the code needed for compatibility is marked by using this library. I only have to prep for 6. Although this step automatically changes the code, we had very little problems with the changes it created. We were happy about this first step running automatically,
22:44
as it was clear the next ones would require manual work. There is an alternative to modernize called futurize. It adds a dependency to a library named future. I personally do not like this library. It seems not as lightweight as 6, which provides only a single module. Future has many
23:08
modules and when looking at the imports, you're not always sure are you importing from standard library or are you importing from the future library. It feels like future is not made as
23:24
a temporary dependency which will be removed after the migration with ease. There are some problems which cannot be automatically fixed, but they can be detected. For instance, in
23:41
Python 3, a class providing the method dunder eq for equality comparison has also the implement dunder hash. Pylint is a tool to find such issues. It has a mode where you can detect them. You need to install Pylint, a Pylint version older than version 2, on Python 2 because newer
24:08
versions are Python 3 only and no longer contain the needed checkers. It will tell you about some problems that even your tests will not find, but they can cause trouble when running the
24:23
application. And now the heavy lifting follows. Run the tests on Python 3 and fix them until they pass. There might be even import errors which already disturb the test collection. Many of them will be due to relative imports which are no longer support on Python 3.
24:44
Running the tests will show up all the problems modernized and Pylint could not find. For instance, stringio versus bytesio. And if you did from dunder future import unicode literals and need both in both Python versions native strings, you are forced to remove this feature
25:04
future switch. Ask a web engine of your choice, a web search engine of your choice for specific problems to encounter. There are already solutions for most of them. I added two example pages in the links section on the last page of the presentation.
25:26
If you are using pytest to run your tests, the plugin pytest-currentn can come handy. It allows to curate a list of failing tests. This way you always know which tests you still
25:42
have to fix. And you see if you broke tests by your changes which were successful before. You are even able to commit the current chain list to a repository so multiple developers can work on fixing the tests. Make sure the tests still pass on Python 2 so you
26:06
are always able to deploy it to production. In Union CMS it was really helpful to add comments to the code branches which are only needed by a single Python version which were
26:20
specific for a Python version by sharppy2 respectively sharppy3. This made it easier later on to remove Python 2 compatibility code which had nothing to do with the six library.
26:40
Okay, step five. Switch to Python 3. This means run your application instance against an at least fairly empty database. This way you make sure the instance starts at least. Sometimes we already had it in our company the tests run fine but the instance didn't start.
27:04
You might find some unicode with bytes issues while clicking through the application. Maybe you should write some additional tests for the problems you find when the application runs. It might be time to migrate the data. Union CMS uses the ODB. This database stores Python pickle
27:28
objects. Python pickle objects behave differently on Python 2 and Python 3. If you store an object with an instance of str on Python 2 it can have an arbitrary encoding as it is actually bytes
27:45
data. But if you read this pickle on Python 3 it expects the data to be UTF-8 encoded as it expects str to contain text data. So you have to convert these pickles.
28:08
If you are using a relational database you are probably lucky and able to skip this step. Now you can deploy the application to a staging environment to test it in-depth from end to end.
28:23
Even with 100% test coverage some issues might slip through. Union CMS was tested by the editors who use it in their day job. They even wrote a test manual for an external tester to test the CMS on the different installations because it was too tedious for them to do this.
28:45
Be aware that the testers will find bugs the automatic tests did not find. Additionally, they will find bugs which already existed before the Python 3 migration was started. You have to think how to deal with them. After a successful test on staging
29:06
it's time to roll out to production. Maybe you can do a canary rollout. This means let only a small fraction of your users hit the server running on Python 3, fix the
29:22
occurring errors and then increase the portion of users which see the Python 3 version over time up to 100%. In Union CMS this approach was not possible. As I said we had to convert the database to be usable on Python 3 so it was no longer usable on Python 2. So we stopped
29:45
editing in the CMS for a weekend and converted a copy of the database and went back online after the conversion. The rollout went smooth thanks to the many testers in the staging environment. There is one final step, drop the Python 2 support. An application can only run on a
30:05
single Python version. After the successful development on Python 3, the Python 2 compatibility code is no longer needed. The code branches for checking for the Python versions might cause at least some performance. Most of them will be done on import time but some of them have to be
30:25
done on runtime. And comments in the code about specific Python versions might distract the developers. It seems to be the best way to drop Python support in a coordinated way instead of when editing the code the next time it is touched. There is a tool called pyupgrade
30:49
which can be used to remove most parts where the library 6 is used. So this can be done automatically. Additionally it converts, it is able to convert code to be modern Python 3
31:02
including the usage of fstrings. After running this tool you can remove the imports of 6 which it does not do automatically, at least not the version we used. And you can clean up the places you might have marked with py2 or py3 as I suggested. Do not underestimate the step.
31:22
Even reading through the diff when removing Python 2 compatibility is a lengthy task. Don't forget to update the dependencies to the newest Python 3 only versions. Some words about the time schedule. We use the following metrics to compute the effort for
31:43
union CMS to Python 3. We at first ran Python modernize and afterwards pylons with pylons without checking in the changes. For each warning pylons showed we calculated 30 minutes to fix it because these are the hard things. In most cases we needed less than the calculated 30 minutes.
32:07
For each doc test we calculated 20 minutes because some tests had to be rewritten as unit tests to support both Python versions and to understand them. We
32:22
knew this from previous projects. These tests which had to be converted to unit tests took more than the calculated 20 minutes. We calculated 5 minutes for each Python module including test modules and 2 minutes for each Python module Python modernize had touched
32:43
to maybe fix the mess the tool created. Plus 135 minutes as a base effort per installable Python package. This means a package with a setup.py which can be installed via pip.
33:00
These numbers added up to a budget which was more than enough to migrate the code. We created a plan for the whole migration. We planned 19 working phases, one iteration each month with two weeks of implementation and testing. So there was still time for bug fixes
33:22
and new features outside the migration project because we the customer and we did not want to stop everything only for this migration. Be aware this 19 working phases were not only the migration from Python 2 to Python 3 but we also migrated from ZOPE 2.13 to ZOPE 4 as
33:43
the underlying main dependency as the application server. Additionally we fixed some technical depth as we are working on the code. And actually we deployed a version running on Python 3 after the 15th iteration so we were a bit faster than expected. We originally planned to roll out
34:08
after each iteration so after each month but the customer was not able to test the whole CMS that often because they did not trust our automatic tests so much. So we had a rollout every
34:25
three months. Each rollout was shipping the current status of the Python 3 migration as it was still Python 2 compatible. Even the big first rollout on Python 3 could have been switched back to Python 2 if it would have failed utterly on Python 3. So the bug fixes would have been
34:46
deployed but on Python 2. Now I'm going to give you some tips and lessons learned. The migration project might take a while. So aim to use a recent Python 3 version. So you do not have
35:05
to start the next small migration project the day after the rollout. Python 3.7 seems to be a good starting point. Newer Python versions might make it a bit harder to still support Python 2. There are discussions to keep it easier to run on newer Python 3 versions
35:29
but it might be a problem. Merge often to the main branch. So do not use long running branches.
35:41
I suggested that your code will run on Python 2 and Python 3. So there's no need to keep it separate from the main branch as it produces enough new problems. Go the route of the small steps. Your project might seem small enough to directly migrate to
36:04
Python 3 without the intermediate steps of supporting both versions as I suggested but we did this on a project, a seemingly small project and we got burned. The changes of updating the
36:21
dependencies to Python 3 together with the update to Python 3 of the code were too much to get it running. So we had to step back, port to the new versions and then analyze the problems in Python 2 as we knew this was a running version which was running before and then we
36:47
could port again to Python 3. So it costs at least time and money. The Python 3 migration of Union CMS took about one and a half years. It could have been done in a much shorter time
37:04
but we wanted to be sure that it was always in the deployable state and features and bugs fixes were always possible during the whole migration project without breaking what we already achieved in the migration process. If you are still running Python 2.7 or even
37:27
older in production it should not matter. If you keep doing so for some extra month to gain the extra certainty the migration will not break everything into pieces.
37:40
That's all. Thank you for listening. I hope you enjoyed the presentation and have some nice takeaways for your own projects. Well thank you Michael, thank you very much. First of all I'd like to share the result of the polls. So I just finished now and we have 38% on somewhat prepared, 38% on highly prepared and 24% on not prepared. So I think our audience is a little
38:07
bit different from the one we showed in the beginning. There we go. You can have a look at the results now. Okay thank you. No problem at all. And then we also have a question from Ivan. So Ivan is asking how would you balance the involvement of QA people, QA engineers,
38:27
testers, test automation engineers during the process of rewriting the service application. How would you find the right balance between enough test validation without slowing the overall progress but avoiding the typical scenario of leaving all the testing to the end?
38:46
Okay that's an interesting question. We were a team of about five people so we did not have such many job titles and the idea was to test each step automatically and by the customers.
39:13
So there was no QA team. So we are not such a big company so I do not have any experience
39:23
of how this would work in a bigger scenario. Perfect, very good. And then we had Andy asking is there any tools for adding tasks for otherwise untested microservices?
39:46
A tool which automatically writes tests. This would be nice to have such a tool. There are ideas. I think that even a talk on this conference about how tests write
40:06
nearly write themselves I didn't attend it. So the way you can do it you can use the code and write tests for the expected values and for the code branches especially for the the happy path
40:27
and then maybe add some for error handling. Perfect, thank you. Then David is asking did
40:41
you run any automatic tools on the complete code base and then hand out the parts to fix as work items or did you run it file by file as work items? We ran it on each package. Union CMS has about 30 pip installable packages and we ran the automatic tools on each package.
41:04
Which are quite big so it took a relatively long time to run them read through the diff and commit it but it worked relatively well. Perfect cool and then last question for now
41:25
I think yes is from Keith. So how did you prioritize developer time between new feature development or bug fixes as opposed to the effort on the migration? Was the pressure from management to pause or delay your work on the migration for business reasons?
41:48
We prioritized by blocking weeks in the calendar for doing the migration so the idea was to have the migration at the beginning of the month so that the other weeks are free
42:04
for bug fixing and development and we did not feel too much pressure to delay the migration because it was important for the customer to get this to get sure they are running on a secure
42:26
environment which is updated to the current Python versions which they wanted for security considerations. Perfect and that is the end then of our Q&A
42:48
and also the end of our talk Michael. Thank you so much. Thank you for coming and sharing us some thoughts on testing and migration from byte 1.2 to byte 1.3. Thank you. It was a pleasure.