We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Growing a Python Community at an Enterprise Scale

00:00

Formal Metadata

Title
Growing a Python Community at an Enterprise Scale
Title of Series
Number of Parts
130
Author
License
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
The talk will showcase how the Python community started and took off in Bloomberg, and how it can serve as a model that can be used within other companies as well. This includes details on how Bloomberg's Python community got started and grew to the place that it is today, as well as tips from its leaders personal experiences.
Scale (map)Enterprise architectureMultiplication signShared memorySoftwareSoftware developerTouchscreenLibrary (computing)Enterprise architectureComputer animationMeeting/Interview
Focus (optics)Group actionMaizeInformationMachine learningComputer networkMessage passingSoftwareSource codeCASE <Informatik>InformationSlide ruleProduct (business)SoftwareWell-formed formulaRight angleSpeech synthesisRadical (chemistry)Software developerSource code
MaizeAcoustic shadowMonster groupService-oriented architectureGame theoryMultiplication signCodeUtility softwareSoftware testingProjective planeService (economics)Level (video gaming)Self-organizationCivil engineeringRadical (chemistry)Game theoryScripting languageShared memoryProduct (business)Video gameData managementMonster groupPoint (geometry)Wave packetEnterprise architectureSoftware developer10 (number)Library (computing)Greatest elementVideoconferencingFormal languageInternet service providerLogicSoftwareRight angleWritingInterpreter (computing)Line (geometry)Service-oriented architectureSimilarity (geometry)Case moddingTerm (mathematics)MathematicsMUDDependent and independent variablesComputer animation
MaizeEvent horizonNP-hardData managementSelf-organizationHuman migrationProduct (business)Multiplication signInterpreter (computing)SoftwareLatent heatEvent horizonNP-hardService (economics)Software testingDatabaseCodeReading (process)Formal languageRight angleBuildingWebsiteProjective planeStandard deviationData managementPoint (geometry)Disk read-and-write headDatabase transactionVolume (thermodynamics)Library (computing)Instance (computer science)Open sourceProcess (computing)WritingAxiom of choicePresentation of a groupMessage passingNumberSelf-organizationFile formatIterationMereologyControl flowMathematicsData structure
Self-organizationPresentation of a groupProjective planeDependent and independent variablesSoftware developerVirtual machine
Right angleFocus (optics)Group actionTerm (mathematics)Total S.A.Dependent and independent variablesComputer animation
Term (mathematics)CASE <Informatik>1 (number)Computer animation
CodeFingerprintEvent horizonEvent horizonGroup actionFocus (optics)Integrated development environmentWave packetCodeOnline helpSoftware developerProjective planeComputer animation
Open setNewsletterGroup actionNewsletterFocus (optics)CodeData conversionSystem callOrder (biology)Lattice (order)FeedbackProjective planeJSON
MereologyService (economics)Right anglePhysical systemMeeting/Interview
EmailElectronic mailing listMultiplication signCompilerGoodness of fitCodeWave packetProduct (business)SoftwareSocial classPerfect groupSoftware testingContext awarenessShared memoryMilitary baseSlide ruleBasis <Mathematik>Meeting/Interview
Gamma functionProcess (computing)Multiplication signEvent horizonSelf-organizationRepresentation (politics)Service (economics)Projective planeTheoryHand fanMeeting/Interview
Projective planeSoftware developerMultiplication signData managementOnline chatMeeting/Interview
Computer virusEnterprise architectureGame controllerMeeting/Interview
Transcript: English(auto-generated)
We do have two speakers this time. I don't know if, what's your setup actually, I didn't have time to ask. Do you have two screens, two sharing or one? The first speaker is Mario Corchero.
He is a senior software developer at Bloomberg. He leads a Python infrastructure team in London, enabling the company to work effectively in Python and using company-wide libraries and tools. And your background is actually Python and C++. Then we have Mariana Polatoglou.
I hope I pronounced it correctly. She is a senior software developer in Bloomberg LP in the mobile team and also a Python enthusiast. And she does the thing called the Python Guild. I think we will hear more about this. So please start with your talk,
growing a Python community at an enterprise scale. Thank you. Thank you very much. Well, yeah, so basically we're gonna speak today about what's Bloomberg and how Python was growing at Bloomberg. It was quite a long journey and we hope that this is helpful to you
if you plan to grow any technology within your company or if you're thinking to trying to introduce Python within your company. It was quite a long trip for us and we hope that this is helpful for other people. So first of all, can I have a show of hand of who knows what Bloomberg is? Okay, okay. That's the only joke for today, don't worry.
So in case you don't know what's Bloomberg, I wanted to put a couple of slides together because at least for me when I joined Bloomberg, I remember this was that I knew Bloomberg as the TV channel that you would just skip through when you were trying to find the channel that you really wanted, right? So Bloomberg is more than just a TV channel. There is this product that we sell which is called the Bloomberg formula.
And basically this is a software that puts together all kinds of financial information on the world and allows you to sell stocks and see how they progress and anything that you might want around financial information. Bloomberg is extremely, I was gonna say old, but let's say elder.
It was formed in 1981 and there are a lot of, we have a lot of employees and we have more than 6,000 developers. The reason why I'm saying this is because I want you to understand that when I'm speaking about how Bloomberg started to embrace Python, this is not a small startup
that you can change in a day. This is a huge elephant that was mainly doing C++ and was used to having the whole workforce within the company to know how to work with C++ and some many years ago as well, Fortran. And introducing Python at Bloomberg was quite a challenge.
I think that similar to what happens in many other companies, Python started to flourish at Bloomberg as a language that was useful for testing and doing some basic scripting. This was in the first company in which I was working, which is Amadeus, this was also something similar. And given that Python was kind of easy to write,
it was seen as easy to write, it was preferred quite often when people wanted to do some test, some kind of like high-level testing or some scripting. I think this is something really important. When you're trying to introduce something at the company, I think it's quite key that you want to start with something small and see how it flourishes and how useful it is.
After writing some really basic scripts, something which was key in the history of Bloomberg happened, which is that a monster was born. There was an intern who created a project called BBPy. BBPy stands for Bloomberg Python, and which basically was a fork
of the interpreter at the time, Python 2.5 or Python 2.6, which was statically linking all the, well, the main C++ libraries that were being used within Bloomberg that allowed Python, well, BBPy, to tap into the solutions that many developers were accustomed in C++ land.
This allowed them to use Bloomberg databases, Bloomberg way of getting the time, how to communicate across services. And this was, as I said, this was an intern that put together this fork of the interpreter. And it was crashing and it was,
all the APIs were full of C++ idioms, and the manage of the lifetime was really C++ wage. But this monster was a really opportunities monster. This allowed the growth of Python within Bloomberg, because then when you were writing your software,
you could actually choose Python to write meaningful code. Before this, you could only use Python to, as I was mentioning, to write some scripts and some basic testing utilities. But now with this monster, you are able to write things that were providing business value.
And I remember back at the time when I wrote my first Python service, my mother will tell me, but is Python even safe to run in production? These were the old days where there were absolutely no support of Python in the company. And it was something new,
but it was not something like enterprising. And something that you might want to take from this is that don't be afraid to be disruptive and create something like a simple POC can get to change a whole company. We've also seen this many times within our company
that I remember there was a tool which was actually written on top of this video by an interpreter that allowed people to write tests basically for their services with basically two, three lines where they were just saying the request and the response. And that grew across the whole community. And that kicked an appetite for more testing.
So don't be afraid to have crappy solutions change the company. This will indeed adapt to your company, but it might initiate a move that it's worth pursuing. And also why I think that Python was really important is the way Python succeed
was because Python arrived when Bloomberg was looking at doing more more SOA. We were trying to create more services and we're trying to communicate it across teams more to our services than libraries. Back in the day, everything would be basically a C++ library unless you need to speak with a UI.
But the problem is like the tooling was not there. Writing a new service in C++ was quite costly. And quite often what you would see is that someone would create the service with like these two example requests and then when you needed to add some new logic that should probably go into a new service
because of the cost of writing a new service that may be a couple of days, people will just say like, look, these are related. So we might as well add them to the existing service. I'm sure everyone who is in an SOA organization have seen this. And but you know the problem with this happens, right? When once you let the mud lurking your service,
you open the door to absolutely anything coming through it. And this is something that was happening in C++. And when we started to show Python to people within Bloomberg, I remember we'd be proud to say that Luke, come and use Python. You'll be able to write the service in just half a day or one day, which at the time it was amazing.
Nowadays, we have managed to reduce that a lot. But this was one of the main selling points of Python at Bloomberg. The fact that you could get a new service ready in half a day. And this is, I think this was probably one of the most important things that Python brought
and what led to Python spreading across the company. We found something where Python was excellent compared to existing solutions. So if you're trying to introduce something in the company, try to think like what's the main value that's gonna be like, if you want to create as well, people like you want people to use it, like what's the main thing that's gonna change
on the life of, well, the work of people that are today using different solutions, right? Maybe you are in an organization that is used like heavily depending on the multi-threaded code. Maybe you want to look at Rust. Maybe Rust is the solution for your company. Try to look for something that will be a game changer
because the existing solutions will have more infrastructure support compared to this kind of like disruptive proposals. And with this, like a community was born. There was no infrastructure support for Python at Bloomberg. I said, this was an intern who wrote an intern
in a team that was doing basically alerts in the terminal. And we decided to start organizing ourselves to share knowledge about Python because there was no material and there was no training within Bloomberg to provide knowledge about Python. We started to do internal meetups. And in the internal meetups, they were a total success.
It was full of people because Python was already spreading. And we were learning about how Python was at Bloomberg and also about things outside. So we're learning about how to use Python, what was coming in Python 3 and things like that. And I think this was also key for Python. Python started from the bottom up
and we had a community behind it back in the old time. That was really helpful because we had a chat that allowed our users to answer questions whenever they have any doubts. And we were actually providing even the support in terms of like libraries
because we were using GitHub Enterprise, but probably any other tool that you have to share code works. We were able to start creating libraries and share them across our users. And this was called Community Project. We were basically, whatever I was writing for my team,
I was happy to share it with the rest of the company. And they started to create more and more community driven infrastructure that started to spread across Bloomberg. I remember it was quite shocking when I saw a really simple library created just for my team being installed in tens of thousands of hosts. I was like, what is going on here?
This is way too scary, right? This is not my main job. How come like this library is being used across the whole company? And this was actually something that you might want to look at because I remember coming from a company where all the infrastructure code was supported by a team. It was quite challenging
because there were some situations, they were minimal, but there were some situations where if something breaks, they were calling me in the middle of the night telling me to fix something, right? And I was like, look, I'm happy to share this code, right? But I'm currently not there and I'll try to fix it, but you cannot expect the same kind of guarantees
from this code as if this were to be my main job. Also, something that we did which I think was quite successful in spreading Python was we set up a lot of workshops. We basically set them up where we're helping and guiding people and holding hands on how to write Python for the first time.
I guess if you're in a company trying to use new technology and people are not comfortable with it, it's really important that you make sure they have a really good experience on the first time they try, right? So we're holding workshops every other week. We're showing people how Python could be used. And one of the main concerns I think that people had
when we were trying to evolve for Python within Bloomberg was that people were worried that Python was going to be too slow, right? We're a company, we're a financial company and a data company that needs to process a lot of data really quickly. And people were concerned that Python was going to be really slow,
but quite often the code that was being written was just only glue code that was interacting with all the services and databases. So that both of them was really the network. And we realized that many teams did indeed have really high volume of transactions, but all they needed was basically
to just have more instances of the service. They needed throughput, not latency. And I think that's something that people really, like that's something that you need to be really careful when you think that you need your code to go faster. Many times you don't need the code to go faster. You need to be able to handle more transactions and you can get that by,
you can just increase throughput by raising the number of services if you're in an SOA organization. Actually the opposite happened. People realized that Python was really fast. We started, we wrote one service in our team in Python and our TL back at the time
was impressed with how quickly we were able to put that together, that he decided to have the whole team trying Python out. Quite quickly, all the teams within the department were impressed with how quickly we're able to put business value into production. So the Python started to spread because of the department and like that across the company.
Like Python spread like a disease within Bloomberg because it was extremely easy to write new code and to adapt to new requirements. So at the end Python turned out not to be slow but to be actually faster. And at this point, our organization were starting to do more and more Python. It started to be one of the standard tools within the company.
And we decided we wanted to engage outside. We started to contribute to Python, to all the open source projects. We were using all this for free, so we had to give back. And we started to sponsor conferences and all the events. We were hosting events and we even started to have a conference hosted in our own building.
And I think I wanted to add this into this presentation because many companies will see this as just giving money away. I think it was super useful for us. We gain on recruiting, we gain on retention and we gain on skill. We were a company that had no history on Python at all. And by reaching out, I think we gain a lot of skill
and we allow our own employees to see that Python was a thing at Bloomberg. And with enough time, there was a point where managers realized that there was no stopping Python at Bloomberg.
So there was time to have a team dedicated to Python. It was important to have a team to be able to tackle large scale projects. We were basically doing everything as individual contributors trying to, in our free time, trying to put some things together. But it was time to have a team dedicated full-time
to do this. And we started maintaining the interpreter. That's the team I'm currently working on. So I'm maintaining the interpreter and then we're working on the migration to Python 3, speeding up the interpreters and anything related to Python within the company. And I was quite touched in 2008
when the head of SI was saying that Python should be the full language of choice when writing new code in Bloomberg. So it was quite a long trip, but I was happy to see that this thing that started just from someone was writing some Python code transforming to this thing. And I'm now gonna pass to Mariana,
which is going to keep speaking about how we structure and how we collaborate. You hear, Mariana? There you go. I cannot hear you though. Sorry, I just clicked a lot of buttons. So as you've heard, we started from an intern writing code
and actually from meetups and all of a sudden with some magic involved and some spells, we got the Python Guild, right? That's great. Of course, there was no magic involved. It took a very active community, people who were very proactive, wanted to see change and actually pushed for it and worked for it,
which is amazing. It took a lot of hard work from everyone in the community, honestly, and everyone who wanted to see this happen. And it takes management buy-in and that is important to say, because the truth is that once you have a Guild,
then you have people dedicating part of their time to something outside of their normal team's work. So the management need to believe that this is useful, this is helpful, this is worth it, essentially. And it takes a lot of iterative reform. We didn't go overnight from the meetups to the format that the Guild is today, actually.
So, Guilds have had different definitions throughout the centuries, really. And even now in different companies where Guilds exist, they have different definitions. So I really wanted to get this right. And what the Guilds are in Bloomberg overall, they are community centered on specific technical topics of interest
and for the advancement of technology in Bloomberg. That means, for example, the Python Guild has a very clear thing. It's about the Python language and the tooling around that in Bloomberg. Of course, there are Guilds like the testing Guild that has to do about testing in general and doesn't have to do with a specific language. And the Guild leaders then are charged
to influence the use of the technology internally and engage with the community externally. So that means, for example, like the meetups that were mentioned, talking about best practices internally and helping out with conferences and the conferences, the participation of Bloomberg in the conferences as well.
So the Guild is organized with having chairs, usually one or two chairs who are facilitating the Guild's processes. So in the Python Guild, I'm one of the chairs actually. And the Guild leaders who are actually making this possible and doing everything else and facilitating the whole Guild, really.
So if you think of the Guild, it's essentially a machine that have a few responsibilities. And those are, first of all, having conference engagement. So that means from looking into what conferences are happening out there and which we can sponsor and should sponsor,
depending with our budget restrictions as well, of course. Helping out with speakers, with proposals, with anything that has to do with conferences, really. Then we have presentations and meetups. This is a lot. You hear meetups a lot. That's because they're great, honestly. They're a very good tool in sharing knowledge.
And with also talking with presenters outside of Bloomberg to come into Bloomberg and present something that's interesting for the whole community and that people should know about. We have internal influence and that goes back to best practices, helping out developers start off with something
or with any issues they're facing on the specific aspect of the Guild. And also collaborative development. So that goes from developing community projects. Also think that the Python Guild is one of the oldest Guilds in Bloomberg and the Python Guild co-exists with the Python infrastructure team. But there are also other guilds
that don't have an infrastructure counterpart and have to do all the development of the tooling themselves. So all that sounds like a lot, right? Because when you get charged with all this, like where do you start? How do you even start? What do you do first? And this has led us to organize ourselves
into focus groups. And that means that its focus group is charged with a specific aspect of their responsibilities. And usually our focus groups are led by one or one to three Guild leaders. And these work great. They're amazing in terms of parallelizing our efforts and separating the concerns as well.
So we have six focus groups in total. And starting with the conferences, this one is the clearest to describe, I guess because I already have, it's the same as the one for the Guilds in general, looking at what conferences are there, what to sponsor, helping out people with their talks,
with proposals and all that. And even with the conference engagement in terms of when people go to the conferences or going to their living rooms in this case and participating in the conference. We have external events focus group. This one has to do with sprints, for example,
as we have in this weekend for this conference and with also other external events as external meetups. We have the co-development focus group. And this one has to do a lot with fostering a good environment in the community and making sure that community projects are able to flourish
also in terms of like what the infrastructure offers and what the end developer may want as a helpful tooling as well. The Guild can help a lot with that. We have the internal events focus group as well. This one is about meetups and workshops.
Our workshops actually have had huge success. They're very good. And we also have the pragmatic recommendations focus group. This one is a very good group because you have best practices, but also you have that not all solutions fit all problems. So you can have different approaches
depending on what the issue actually is. And last, we have the training focus group. So in Bloomberg, the training, when you join the company is fully in Python that explains the tooling in Bloomberg. And this focus group has to do with interfacing with the training team and anything that they may need.
So how we organize, we have open meetings with the Guild leaders every two weeks in which we discuss any issues and we talk about proposals and anyone can actually join, which is amazing to get feedback from the community straight away as well. We have a newsletter sending out a day before the meeting that it's focus group give out their updates
and ask for any call for actions. And we also do retrospectives in order to reform ourselves. This is one of the best agile tools out there. And the most important of all, the community contributors without them, the Guild wouldn't exist and it wouldn't actually even make sense to exist. And it is amazing that we have that.
And also think that the Guild leaders are leading the focus group, but they can't contribute to everything. So for example, if we have five community projects running, not one person cannot do all the coding and the community is what makes this possible and what makes this actually worth it. So what we have achieved,
we went from a main C++ company from like, why do you want, I'm sorry, why do you want to write this in Python? Like what is even Python? Isn't it too slow to, why not in Python? Python sounds great. And that is amazing. We have a very active community ever growing and it's one of the friendliest places to be in and I absolutely love it, no bias.
And we also have a Python infrastructure team which shows how much the company is committed to Python actually, which is amazing. So what's next? We have, we're currently reforming our focus groups and thinking of new focus groups we could offer and should offer. We asked the community in one of our open meetings,
what issues they're facing and would like to see focus groups for. And we're currently in further conversations with the community on that. And of course we keep changing in order to stay current and to actually meet the community's needs and that's what we need. All right, that's it then.
Thank you very much for listening. Are there any questions? Thank you very much for your talk. Very interesting concepts, this Guild system. I see we have questions in the Q and A but I actually do have a question first about this Guild system is how do you motivate people,
internal people to join the Guild and the meetups? And the second part of the question is do people from other Guilds, I don't know what you have, for example, a C++ Guild or whatever, do they join too? So actually I guess that depends.
There are many people who are very proactive about it and for those we don't need to do anything obviously. But of course I think that it goes back to what Mario said in the very beginning that there were a lot of workshops to show people how easy it is to actually write a service in Python and have it out to production straight away.
So it's about convincing people that it's worth it. And then it depends, I guess, on the topics and so on but we've definitely had people who converted from just like writing C++ to writing Python and now I'm being very happy. I don't know if that answered your question fully. Perfect, yeah. And now we have many questions in Q and A.
Oh, we are running out of time. Okay, let me go with the first from Diego. How do you raise awareness about good practices, software quality, sharing tools, designs or techniques? Any suggestion there? I can take that one if you want, Mario. So I believe that's a great question
and I was actually hoping to mention it in the slides but I cut it because I was seeing that that was overrunning in time. We were using, we were basically relying on an internal mailing list that we have and also pushing a lot on meetups and workshops. Lately we also start to have more like proper training
which helped a lot. But this is a really good question and exactly key because when we started and when people started to move from C++ to Python, we saw that many people were not writing any tests at all which for us was unthinkable in Python for a large code basis. But things like people were relying on the compiler
and we had to put a lot of emphasis on making sure that look like you need to follow what are good practices in Python and you need to make sure that you really write tests when you write new code. So like absolutely agree that you need a way to raise awareness about practice and everything. And yeah, we were basically relying on internal
mailing list, meetups, workshops and the internal networks that we have for that. Okay, so we have time for the last question. How do you manage the time of organizing events, leading these initiatives and doing your engineering job at the same time?
How supportive is Bloomberg in this regard? I can take that one as well then. I think you can probably answer as well in front of the same mark. So at the beginning we started basically, so you know, because what we were doing
in some way was also relevant to our team. The beginning there was no issue at all, right? It was kind of like a community project but it was seen somewhere related with our team because we were starting to do more Python services. When this got more formalized, there are some organizations like the guilds, as Marc mentioned, or there's some other thing called like the technical representatives,
which is one of the organizations in Bloomberg, that in theory give you 20% of your time to do whatever you want. That's something I think many other companies do. And so basically we were, I mean, I would like to say that I was just using that 20% of my time to do Python projects,
like I think it would be precious, but we all know that's not true. And at the end I was basically overworking, which is something I don't recommend. But yeah, like Bloomberg in theory is really supportive of this. The company is really supportive. And I think I would encourage every company to do that, like trying to encourage this kind of side projects that in some way I think advance the company.
But it's really, if it's something that you are passionate about, it's really easy to get out of hands. So that was kind of my situation. And Omar, how was we assigned? Sure, the only thing I was going to add is that I think it goes back to getting the management buy-in and convincing them that it's actually something
that is useful for the company overall. So yes, there are going to be projects for the team obviously, but some of the time can go into guilds. And that helps a lot, even with the personal development to be honest. And that is amazing. Okay, thank you again very much. Unfortunately, we have two more questions here,
but time's up. So please take these questions to Discord chat with Control K. If you enter community, you will get the channel talk, Python Community Enterprise. Hello. And there will be more questions. So thanks again for this talk.