Python Not Recommended
This is a modal window.
Das Video konnte nicht geladen werden, da entweder ein Server- oder Netzwerkfehler auftrat oder das Format nicht unterstützt wird.
Formale Metadaten
| Titel |
| |
| Serientitel | ||
| Teil | 36 | |
| Anzahl der Teile | 173 | |
| Autor | ||
| Lizenz | CC-Namensnennung - keine kommerzielle Nutzung - Weitergabe unter gleichen Bedingungen 3.0 Unported: Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben | |
| Identifikatoren | 10.5446/20059 (DOI) | |
| Herausgeber | ||
| Erscheinungsjahr | ||
| Sprache | ||
| Produktionsort | Bilbao, Euskadi, Spain |
Inhaltliche Metadaten
| Fachgebiet | ||
| Genre | ||
| Abstract |
| |
| Schlagwörter |
EuroPython 201536 / 173
5
6
7
9
21
27
30
32
36
37
41
43
44
45
47
51
52
54
55
58
63
66
67
68
69
72
74
75
77
79
82
89
92
93
96
97
98
99
101
104
108
111
112
119
121
122
123
131
134
137
138
139
150
160
165
167
173
00:00
PlastikkarteSoftwareentwicklerTeilmengeApp <Programm>Vorlesung/KonferenzBesprechung/Interview
00:23
Elektronischer ProgrammführerTaskPortscannerDokumentenserverReelle ZahlLambda-KalkülAlgebraisch abgeschlossener KörperQuick-SortCodeDefaultInformationFormale SpracheSoftwareentwicklerBenutzerbeteiligungArithmetischer AusdruckMakrobefehlDifferenteDichotomieFunktionalRankingFramework <Informatik>DatenverwaltungKonfigurationsverwaltungReelle ZahlServerProgrammiergerätBefehl <Informatik>Mailing-ListeNetzbetriebssystemLambda-KalkülTopologieComputeranimation
03:04
Apache CassandraPlastikkarteMetropolitan area networkBitrateTermOverhead <Kommunikationstechnik>ImplementierungProzess <Informatik>AuswahlaxiomDienst <Informatik>MultiplikationQuick-SortProxy ServerSoftwarewartungProgrammbibliothekTopologieVersionsverwaltungMultiplikationsoperatorSoftwareSpezifisches VolumenPlastikkarteMathematikCodeAppletKartesische KoordinatenMathematische LogikTermDatenparallelitätMessage-PassingSkalierbarkeitSchreiben <Datenverarbeitung>BruchrechnungMigration <Informatik>AdditionBitrateTextbausteinEinfach zusammenhängender RaumImplementierungOffene MengeAlgebraisch abgeschlossener KörperServerDatenbankMaßerweiterungOverhead <Kommunikationstechnik>PunktBenutzerbeteiligungLastteilungPhysikalisches SystemFitnessfunktionFramework <Informatik>SoftwareentwicklerOrdnung <Mathematik>HalbleiterspeicherLeckZweiSchlüsselverwaltungTypentheorieCASE <Informatik>Digitales ZertifikatSystemplattformServiceorientierte ArchitekturZahlenbereichInstantiierungDifferenteSchaltnetzKonfigurationsraumFormale SpracheClientSchlussregelKanalkapazitätGüte der AnpassungShape <Informatik>GrenzschichtablösungModallogikZehnZentrische StreckungRechter WinkelSyntaktische AnalyseComputerunterstützte ÜbersetzungGeradeSchnittmengeFlächeninhaltNatürliche ZahlOffice-PaketWellenpaketExpertensystemComputeranimation
11:34
Rechter WinkelProjektive EbeneKartesische KoordinatenMultiplikationsoperatorPatch <Software>Proxy ServerVersionsverwaltungMathematische LogikDifferenteQuick-SortQuaderZentrische StreckungSkalierbarkeitVorlesung/Konferenz
12:12
Metropolitan area networkTotal <Mathematik>Projektive EbeneFormale SpracheSoftwareentwicklerAlgebraisch abgeschlossener KörperKonditionszahlDatenparallelitätPrimitive <Informatik>Computeranimation
12:42
GruppenoperationPrimitive <Informatik>DatenparallelitätAggregatzustandPunktVorlesung/KonferenzBesprechung/InterviewComputeranimation
13:06
Physikalismusp-BlockCodeVerkehrsinformationNetzadresseCloud ComputingInstantiierungHardwareWeb SiteDatenverwaltungPunktwolkeEndliche ModelltheorieServerCodierung <Programmierung>DatenanalyseMereologieURLSichtenkonzeptAnalysisQuick-SortNatürliche ZahlTaskKartesische KoordinatenFlächeninhaltGruppenoperationTabellenkalkulationSchreiben <Datenverarbeitung>Mathematische LogikDokumentenserverVorlesung/KonferenzBesprechung/Interview
15:02
DatenverwaltungWort <Informatik>SoftwareMereologieOffice-PaketDokumentenserverRechenzentrumMatchingEndliche ModelltheorieGeschlecht <Mathematik>Formale SpracheGeradeEreignishorizontKonfigurationsraumInteraktives FernsehenCodeBimodulProjektive EbeneQuellcodeQuick-SortPolygonnetzComputeranimation
17:06
VerfügbarkeitProgrammbibliothekFormale SpracheProjektive EbeneStützpunkt <Mathematik>DatenparallelitätEinfache GenauigkeitCodeQuick-SortProgrammierumgebungStandardabweichungt-TestAggregatzustandCASE <Informatik>Wort <Informatik>TermSprachsyntheseDigitaltechnikComputerspielGradientProxy ServerVorlesung/KonferenzBesprechung/Interview
19:49
Metropolitan area networkProzess <Informatik>t-TestProgrammiergerätXMLComputeranimation
20:26
Produkt <Mathematik>Analytische MengeDatensatzProgrammierungKonfiguration <Informatik>Web-ApplikationQuick-SortAlgebraisch abgeschlossener KörperProzess <Informatik>Vorlesung/KonferenzBesprechung/Interview
Transkript: Englisch(automatisch erzeugt)
00:04
Work at a company called Braintree We make it really easy for you to accept credit cards PayPal and other payment methods online and in mobile apps This talk is not currently on github, but it will be shortly and you'll be able to find it there So let's talk about the title of the talk. I find not recommended
00:23
It might sound like it's a joke, but I mean it quite literally At the company where I work we have something called the radar and it contains information on the technology We use whether you should use them whether you should never use them again and sort of where to look for examples and other relevant information and
00:42
in the radar Python is specifically listed as currently in use but not recommended So this talk is about why we originally used it why it's not recommended now and what we still like it for and what? We're gonna use it for in the future So first some background the obvious question is what do we use instead if we don't use Python?
01:02
There's got to be something else We uses our general-purpose language and the answer is Ruby brain tree loves Ruby brain tree uses Ruby by default for pretty much everything But I promise this isn't a Ruby talk. I Personally am a Python person. I don't love Ruby. So we'll we'll talk about it only as background
01:21
Brain tree has bought into the Ruby ecosystem. We use rails as our web framework We use Capistrano for remote server management We use puppet for configuration management and we use rake for sort of scripting builds and automation so You probably wonder what it's what's it like doing Python at a Ruby shop something I've been doing for three years now
01:45
There's a lot of jokes about Python because it doesn't have quote-unquote real lambdas which people just mean that function definitions are statements and not expressions and people are like just to be able to define a function wherever they want and Even though functionally they're the same people don't like the syntax difference
02:03
It's seen as an elegant compared to Ruby or elixir or closer This is both because of the statement expression dichotomy but also because of the generally structured syntax and Because in these languages magic is really easy with macros in the lisp or with everything and Ruby is magic And it's just a little harder in Python and people don't like that
02:23
It's sometimes gets dismissed as a dying language because of some of the negative publicity Around the slow adoption of Python 3 I don't see it as a failure I don't think the Python community sees it as a failure, but people outside the Python community sometimes see it that way The languages are also similar in a lot of ways
02:41
Almost too similar for their own good when you're a Ruby programmer a lot of people will sort of apply judgments that you'd apply to Ruby code to Python code without taking into account the difference in idiom and So if you were to apply to Braintree as a Python developer might hurt you a little because people are gonna expect your Python code to be written like Ruby
03:00
And it also means that our Python code can be a little less than idiomatic So despite using a lot of Ruby first and foremost we believe in using the right tool for the job So you're not gonna find a ton of this at Braintree although it does happen once in a while So what do I mean when when is Ruby not the right tool for the job? And we've sort of found two main times
03:23
One is when you need the JVM ecosystem You might say Just use JRuby and you have access to everything well we found that that gets a little messy and that It's not really a good long-term solution although. We have done it in the past when we needed Java in the short term
03:41
Primarily we'll need to use Java if some third party who maybe wrote their API in 1999 Only has doesn't have an open API spec, and they'll just give you a library and your choices are C++ or Java well We're gonna choose Java There's also a lot of great tools that work best with the JVM like Apache Kafka
04:01
Which is a message broker for handling high-volume data feeds and Apache Cassandra, which is a distributed database for handling large amounts of data So What do we use when we need the JVM well? Like I said we have tried JRuby Historically along with Java we haven't found it to be that great
04:20
And then more recently over the past maybe year and a half two years at this point We have used closure pretty successfully although. We've still found that there are times that you really want to use Java directly so the other main time that We find that Ruby isn't the right tool is when you need to write a smart proxy now our business is basically to sit between
04:41
Somebody who wants to sell something online and the banks and card networks So we basically are a smart proxy and so our big smart proxies build up of a lot of little smart proxies You can see this is the logo from a party that brain tree and PayPal through it South by Southwest a couple years ago It's smart. It has a brain so it fits
05:02
We have really high uptime requirements We need to be available when there are temporary networking problems when we need to failover services when you have to run database migrations When we get huge traffic spikes like I don't know uber is it's New Year's and uber is running tons of tons of rides And they want to charge people's credit cards And we also have a big problem with the services behind us the banks the card networks
05:24
going down and we need at the very worst case requests to fail gracefully and Preferably if the outage times are short like on the order of a few seconds We want those requests to succeed even though the service behind us failed So we use them to make our outgoing connections appeal appear highly available to our internal services
05:45
We use them to pause incoming requests So that our internal services won't actually see any requests coming in and we can do whatever we want to them But actually clients are still able to connect to us We do custom rate limiting we have pretty complex rules for how much capacity different clients and different types of requests can use and
06:05
We because we integrate with a lot of legacy services We often find that we need really weird SSL or persistent connection configurations And this is something that we've often had to do ourselves We also have very complex. We try logic and a lot of other custom logic that we need
06:24
This is where we've historically used Python It might seem a little odd since handling a large number of requests doing a lot of concurrency is not Necessarily the first place you think of Python, but it was actually a pretty good fit for a long time Specifically because of tornado tornado is a web server and framework for doing non-blocking IO and
06:46
You get the benefits from Python of rapid development and easy to learn and you still get the necessary IO concurrency to handle tens of thousands of requests So back in 2013 Python was in pretty good shape at Braintree It was in use for several of these internal proxies and it served us
07:02
Well for a couple of years and it had several internal advocates not just me but other people too So what happened that I'm giving a talk called Python not recommended The platform really did fail us as we started the scale Concurrency and the framework really isn't enough nowadays we sort of expect our languages to have
07:23
Concurrency built in as Python 3 now does and as go and many other languages do and you really expect that Concurrency logic to not get in your way when you're writing business logic, which we found that the concurrency logic and tornado really does It was also too much work to keep up with changes in tornado
07:42
we we looked at new versions a couple of times and it would have really been a complete rewrite to use the new API's and We really didn't want to spend the time for that and we didn't trust that we would be able to make those changes without breaking anything And so because we were using an outdated tornado version maintenance overhead Could really be pretty high. You can't Google things easily because people have moved on to newer versions
08:05
It's hard to find the right docs and you end up in callback hell because you're still using old less elegant API's we found that logging actually has an Unsettlingly high overhead every time we line logged a line. We saw requests pause and
08:23
Eventually those pauses added up to enough to be really significant as our volume scaled and then there's no SNI support Python 2 historically SNI lets you serve multiple SSL certificates in the same server port a lot of our customers used it and
08:40
It was only very recently introduced into Python 2 But the version of tornado we're using is so old that even if the Python supports it the tornado version doesn't The applications that we wrote also really failed us and a lot of this ties in with those platform failures But it's also to a large extent our fault The smart proxies were really too smart
09:01
the logic was all mixed in with the concurrency boilerplate making it hard to understand and They were trying to do so many things that the code ended up That was meant to do one thing was coupled with the code that was meant to do something else So when we tried to rip out the code to do connection pausing it completely broke rate limiting and we had to put It back and leave this completely now unused code in the in the code base
09:22
Straightforward Python implementations as we scaled were not fast enough So the rate limiting code started to add an unacceptable amount of overhead to every request that came in We also found that in addition to being too smart the proxies were not smart enough We couldn't just write certain pieces of business logic in the proxy the business logic had to be duplicated in the main application and
09:46
That's something that We really don't like we don't like writing the same logic twice And none of these applications were really built for horizontal scalability They all assumed that a single or in some cases a small number of instances would be enough forever
10:01
And so they weren't designed for us to run ten or a hundred of these they were designed for us to run two or three So what solutions did we switch to? What did we what made Python obsolete in these areas? So for our incoming request proxy? We've switched to a combination of nginx and H a proxy along with PG bouncer
10:23
So in nginx and H a proxy were able to do approximately the same complicated rate limiting logic and load balancing that we were able to do Previously with our proxy in a pull layer and We've also moved pausing completely out of the proxy layer and into PG bouncer
10:41
Which is basically another proxy that sits between the applications and post SQL We Then wrote our main outgoing proxy as In node.js, but this was actually a failed attempt. It had all the same types of problems as Python It was still trying to write our own tool to do a job that
11:00
We weren't experts at and we had problems with failed persistent connections and with memory leaks in node that led us to abandon it and move to nginx and H a proxy now the key here was timing is that H a proxy 1.6 had features that we really needed and So we've we've now moved to that and it allows us to remove another custom piece of code from our system
11:23
Finally our sort of most complicated outgoing proxy We've decided to rewrite to enclosure using Apache Kafka and it allows us to centralize all the logic in a single application We can build it pretty easily to horizontally scale almost linearly and of course you get SNI support with the JVM right out of the box
11:42
and unfortunately, we canceled that project because it wasn't a high priority it was gonna take a lot of time and So instead I wrote a monkey patch of tornado to support SNI, even though it's a really really old version tornado So it's not great. We still have logic duplication between different applications and we still have
12:01
Sort of lack of horizontal scalability, we basically run two of these proxies, but for now it's okay, but the problems are unsolved So as of late 2014 All the smart proxies were on the way out Or in use but not because we wanted to Not recommended for new projects. It's official. It's in the documentation that we use at Braintree and there were fewer internal advocates
12:27
Now this isn't because all the Python developers got mad and left Braintree But a lot of the people who used to really like Python Now have moved on and prefer languages like go or closure or elixir
12:40
Some of which have more in common with Ruby so you can understand it and some of which just have better concurrency primitives than Python too Just to be clear. I don't fall into any of these groups. I still like Python and still use it outside of work so This is kind of sad and it makes it really sound like the state of Python at Braintree was really sad
13:01
But that's because that's the point of the talk We've definitely there is something we will never use Python for again, but actually things are looking up overall We're now using Python in areas where it really shows its strength instead of just sort of as the Swiss Army knife glue code of our code base
13:20
So the first place is data analysis, this is probably surprising to exactly no one Our Business analysts have really used it to replace Excel to write sort of one-off reports and do Smaller monthly tasks that don't really need to be automated our data analysts have moved more and more from writing giant crazy blocks of SQL to putting more logic in Python so that the code is more maintainable and
13:45
Understandable to a larger group of people and these are people who've maybe done a little programming before but are really buying into Python Very rapidly, which is cool And finally our data scientists are really using it to replace our Part of that is because of the great modeling and analysis tools that Python now has
14:01
But primarily it's because it's a lot easier to deploy your solution It's a lot easier to do the sort of ETL steps that happen before You do the modeling in Python than it ever was in our Finally well, the next thing is really infrastructure management and this is somewhere that historically we bought into puppet
14:20
Wholesale we have huge repositories full of puppet code puppet is a ruby-based tool but recently we found that Python has a really good niche here and We use it to manage certain resources like IP address physical ports server locations This is something we actually didn't do with puppet. We did with a bunch of Google spreadsheets
14:40
so the centralized application with a lot of the like Better views into the data is really helpful We've also used it to manage cloud instances because we run our own physical hardware for a lot of things We had a pretty unsophisticated setup for managing cloud infrastructures. Namely. We used the User interfaces we basically log into the website and would start and stop instances
15:03
Now that we're starting to do more and more things automatically it needs scaling Excuse me use scaling we Had to have an automated tool and Python is the right way to do that And finally we use our switches for
15:21
slightly complicated setups we do try and make sure that even if switches fail in our data center everything keeps going so everything is all the networking is mesh and The switch configurations are pretty complicated and we didn't find a good way to manage them with puppet and we found that Rather than writing a custom puppet module, which is can be pretty complicated
15:43
It was actually easier to emulate what puppet does in Python and pull from the same data source that our puppet repository does And so all the code is in Python and it's like a hundred lines rather than writing a very complicated puppet module that we'd Never be able to change because no one understood how it worked The Python community is sort of the final thing
16:04
It's a big advantage of the language in my opinion and something that has been very beneficial to Braintree. We host a lot of Python Meetups we have a monthly project night. We host Chippy the Chicago Python meetup about twice a year. We've done a couple of events with pi ladies now and one with Django girls and
16:24
We've also sponsored other events outside the office and this has really helped with our hiring We've hired several people now who first heard about Braintree through these events and came to Braintree even though they don't get to write Python because they know we support the community and Recruiting is one of the hardest things we do. So this has been super helpful
16:43
We also find that giving talks is a great way to spread the word about Braintree and I'm not the only one giving Python talks one of my colleagues has given a couple in Chicago and Has given one with me at Northwestern University Our customers also use Python. So having us support the Python community makes them feel more connected to us
17:03
Some of the biggest startups in the world who are customers of ours Have pretty large Python code bases and it's much easier for them to keep using Python to connect to us and that includes as they migrate to Python 3 we have our Python library is Single code base Python 2 and Python 3 and a significant amount of Python 3 traffic comes through our API
17:24
So having that and supporting sort of the next the future of Python has helped us gain and keep merchants so Now we're at 2015. So what's the state of Python now? Python 2 is definitely showing its age internally and in general especially around concurrency. I don't think anybody really disagrees with that
17:47
And as the standard tools like HAProxy and nginx improve more and more We're sort of losing a use case for Python being the jack-of-all-trades The language you can use to write whatever tool you need is A little less important as the standard tools tend to be there for scaling and for high availability
18:03
Data science is really important for pythons future I think it gets the foot in the door for Python at pretty much every company Which is a great way to keep people who like Python interested and to sort of keep it in your mind for When to choose it for other projects The community is also really important. It's been great for us at Braintree and
18:25
it's just One of the reasons that Python is as successful as it is Thank you, that's all I have. I'm glad to take questions.
18:52
Thanks. First great talk. Thanks a lot. I have a question about like if you See yourself as a tech hub more or less if you have some... I'm sorry. I'm having trouble understanding you
19:03
I'm sorry. I'm a little hard of hearing so you're gonna have to speak up Thanks, and the question is about if you have any Interns, I mean people who like students or such Walk into your company. Are they keen on learning Python or something more? How does it act in your community?
19:26
I'm sorry. Could somebody else repeat the question? I just had trouble understanding it. Is it advice for students About choosing Python or something else? If they come to your company or your like environment Are they keen on learning Python or something else like Ruby or stuff? So what do they do? What do they choose? How do they act?
19:47
Do they choose to learn Ruby or choose to learn Python? Yeah, yeah. So I think that We definitely Look for people who don't want to do one specific thing We look for people who want to learn and want to use whatever tool is best for the job at Braintree
20:04
So I think that a lot of the people we hire and a lot of the students we talk to Are people who have maybe done Python in school, but are open to learning anything I'm not sure if that answers your question
20:23
The the non programmers or non professional programmers like the the analysts you talked about did they choose Python because they tried other options and like closure and didn't like them or Because they already knew Python or because you suggested it or what was how does it find its way into their their job? So
20:42
The sort of the guy who started our business analyst analytics team He'd been doing a lot of our reporting manually in Excel for years and he decided to Learn to program and so first he actually learned Ruby and he wrote a web application that we used in production for years