Merken

Building RailsPerf, a toolkit to detect performance regressions in Ruby on Rails core

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
the and and and and and and and and hello my name is therefore trough i the from
far I am Filipino I work on Capistrano as maintenance and I'm aroused committed to I work at the rel shop
called elections we do the rail development consulting and we work we developed new on Russia since the early days we had in a lot of work for you which is my current projects and my topic today is
about performance regressions interest yeah I'll talk about what is the performance regression how to test and how to solve them or whatever tricks to avoid them and about the 2 I've been working on the last few months to gage a performance reference in rails automatically under about the state of this so in the future but yeah before we start that define what is that performance aggression performance aggression is a situation where the software still functions correctly but it takes more time on to compute or it consumes more memory when compared with previous versions and what is the performance aggression in males imagine as situation where after another great to a new version of rails fresh new version Chinese of the Bayesian it is in prison now from 250 ms to 4 seconds that's quite huge right that's exactly what happened in 3 . 2 . 15 because of some Bogbean sprockets and that no I have a story why we should care about the farmers regressions funerals and why is it a problem for all of us
right after the rails 4 . 2 Release Candidate 1 was released that is
costing they have quite a huge ad they decided to upgrade their on these Release Candidate 1 and just try it rose 4 . 2 because they were really excited about all those the at record patches by error and they expected some performance boost from adequate record but what they found that actually is that the this rose 4 . 2 Release Candidate 1 was 2 times slower and then rose 4 . 1 that was very surprising and there was an performance aggressions obviously luckily this was just Release Candidate 1 and after applying a lot of patches on rails that by discourse demon by resting on the main Grails 4 point to that of the last release candidate number 3 as fast as rail 4 . 1 and and even a bit faster OK but now it seems like a pain in the ass for commuters growth inhibitors but not for use all the variables who have who only have some small or big Orelle's and what is the problem for everyone because faster the fastest framework the faster is your app and by speeding up on act at to record we've had a plate record patches and letters and made your apps faster just because at to record is usually tied with the typical Rails at yeah yeah to get a better understanding of I have some example from URLs comets affecting performance and solving their performance Russians of let's review them but before we do that let's think about what kind of metrics that we count on when we talk about performance regressions in rails these 2 metrics are timing and allegations what is timing timing is the amount of time some method or some piece of code takes to execute somebody decent diamonds dining we make our methods faster and we make our ap here is an example of code so very small example of how to measure the timing is quite fast straightforward so and I want to reduce number of allocated objects been Ruby inhibit their memory because garbage collection is by the expensive operation in Ruby and by allocating less objects really less work for a garbage collector here is an example of how to measure a number of objects allocated in your code we get statistics from members collector and then calculated and most of a lot of performance issues can be solved just by keeping in mind as of a set of traits how do not basing better primitives in Ruby works of babies under but this better print basic primitives I mean strings hashes blogs Paris and so on so we will start from comments from rails that are working with strains of having some issues with strings 1st comment and by and Patterson now I assumed that if you have the so the problem of over this code was that it allocated at the problem of this code was that it used last methods to concatenate 2 strains while in fact and interpolation is way faster really fast faster another problem of this thing solved is that we don't other allocate 1 more string before it concatenate them we just using interpolation once here's another example of my sense friend who is actually in this room and so the problem was that the code was called on every Active to record tributary I imagine that I read reading attributes is the core functionality of there at a record and on every atom trade we allocated in history and by moving out of the string into the constant freezing this constant and Sam produced a a number of allocated objects on the discourse from that and by 5 thousand that's 5 thousand objects just by moving out 1 constant 1 string to to the constant the here's another comment is not from Borell's itself it's from their act jam and Centrale the euro but as you know the red gem is use arousal odd so the problem of this piece of code was that it used outlook whenever requested collocated new variables U strings like content type of them and there will many of them so whenever a new request we allocated a bunch of new strains and then use them but using the same freak I described in the previous slide moving out these strings into constant reviews number of allocated objects on broke 1 request to write by 40 % 40 % that's almost a half of all objects inside the jam reduced and just by moving out this constant and this was 1 of the popular Wordpress and it have another example and it is from the active support but it is of it's working with them strip of time formatting and just replace the time pattern we used just some 4 times allocated under strain every time but in fact we could use only 1 redirect to regular expression what to do all the same things so we don't need for steps right just 1 regular expressions the and this is the comments by my colleague at arrival of in the family of bring on on but the thing was that the in group B 2 . 2 in previous versions of rails we use regular expressions for string replacement out like this i'm because replacing it with a regular expression was faster in Ruby 2 . 1 but it it's interesting case because in orbital point to optimal replacing strings with strings is now faster 1 . 5 times faster and that this quot string method it is called on every when every while when you make and you square the so if you have let's say 100 queries on some it's called 100 times of and now we have some examples working with fascist
and is a piece of code from neural for and to override just 1 has options we allocated new area and called merge just to override 1 half value but as you can see we can just use that have set method without allocating Harry without using margin and then use the same trick by and Patterson but we don't need to as this that we don't need to allocate new hash we just said the value of the same cash without marriage of just right this year so and has done a lot of such performance specialists in at record and is in a great job that that the and then 1 more example of about working with fascist and this piece of code and is from a to record connection adapters and it was called time from there and then when we wanted to iterate over all hash values we allocated new area with all hash values by using their bellies methods and then we iterated over it but if we think there is a method called the stallion not to do the same thing and by using this method we can avoid allocating you area in memory we can and this area may be quite big if a has is good to remember that we want to optimize only called code by hot code I mean the code which is called frequently on a request you many times on every request and the rails code is like a code in the rails in the framework is not usually of the coding rails these hot because it's framework used by thousands and thousands of what different press applications there is it makes no sense to optimize that piece of code which is called once a month from a background job defined benefit much right and now we have some idea what kind of performance changes I mean I here my talk and what if we track these changes to see their roles performance the overall growth performance this idea to be of service to attract rails performance changes was suggested in their as best camp and I became really interested because it's a great idea about how we can optimize and how we can save time for growth inhibitors of listed to attract and to track these changes made some kind of benchmarks on 2 benchmark that the coat other any existing benchmarks the answer is yes there is a great benchmarks
right inside their discourse application and this is not a rails something like this is just a benchmark of this forest and discourse app is using the rest of the framework so in this it is then by
discourse is setting up the database populating the database with records starting on a unicorn in production environment and making a lot of a lot of requests using that event and then of the this benchmark brings timing and memory usage it said Ray Ban part it is used it was used in by she want to optimize garbage collection algorithms in MRI and these benchmark is used every time this course upgrades on URL version to see if this resolution is faster or not it's a great approach as I sense but we also want to measure of performance of phrase components separately aside
from making the least of these components we can see it on my slide ended I didn't include access support here because at of support is something very stable we don't change much and it's almost frozen and bottom parts the change a lot at to record actions controller Action view I wrote some benchmarks run
them against a few major rails versions because this versions of my slides and the the 2nd the and I visualize
this common base of benchmarks on
that on the static page using the 3 JavaScript and JavaScript framework and I will give a link to these static page and later on he considers benchmarks there the higher is the bot their batteries performance we use iteration per 2nd here to measure on
you can see the trend call rails becomes faster and faster right but now I'll talk
about what these than mice are doing so
here is an example of benchmark of Active Record finders weak rate 1 record which right 1 record in the database that user records and then we are trying to find this record of using the find method so when trying to find it actually 100 times this is what we call Meeker benchmark but you can see some special help called benchmark the trails and let's check out what this helper is doing inside so these helper disables garbage collector if it's necessary each warms up the code that is really important to warm undercoat to warm the interpreter before we run an hour however in Marx then it makes some number of iterations this number can be a hundred or thousand or a few thousand and then we calculate the mean time in an object collocations those metrics I mentioned before here is another example of the benchmark off craic methods in at a record of weak rate we crave 1 record inside the butterfly block and then see the result I know I have a story why
is it so important to have a acquired benchmarks and to run it them in i in that production the name environment close to production here is a
fool at benchmark by full and I mean some small riles and we few controllers few models and we are in this benchmark where making your request this have here the final version and we can see that for L 4 . 2 is it it faster than some Primož versions but the 1st version of this landmark look like this but obviously I didn't believe it because it was some mistakes and I decided to discover what's happening inside with a
tool called prof and this is still made by M and looked at this is a tool to our city of called inside there respect and see some statistics about these calls but what I got here is an
example of the difference between the calls in 4 . 1 and 4 . 2 and what I
discovered is that in their environment of these applet run was not close to the production environment that we have in rails and Our after fiction it on the benchmark started to look at
closer to the real life right no we can have some set of benchmarks and it's time to automate trying them against new versions of France we love automated
things in the community right but there
are great examples of how to maintain my examples like Travis that automates build metrics and fast and Hounsell that automates the CodeView and automates review of the code for review guidelines so we can be of some
service to the ultimate trying these benchmarks against new versions new promise of France what do you want from the service that does as I said I run benchmark for every you comment in rails run benchmark of for every new pull requests and report the results of this benchmark to the poorest 1st author and to see some trends in resources the charts I started from making a prototype of the service of it doesn't generate it was very small and simple it could make some of things like fetching you comment from Reston running to only 2 benchmarks against it because and generous on 2 benchmarks now would have more than at some point of felt like and building yet another Travis sided with all those infrastructure and a lot of code just like reinventing the wheel and just a month before I started development and there was a service called would mention made by some suffering and Alan please upgrade your hand if you thought about revenge so I think nice but for those of us who don't know about the event yet revenge is a
service to automate training and benchmarks for Ruby for MRI so every comment to MRI did have it France new benchmarks but it has a nice infrastructure problems with Dr. and it
has sponsored hardware because it's extremely important to run your benchmarks on a dedicated hardware because if you run these benchmarks compliant the CPU being in cloud is shared between many instances and if someone else's mining bitcoins on another instance it may affect your on benchmark results which is that and after getting contact with their revenge on all 1st of all we decided that I will just in that suffered into revenge and there are no lights see how it works on
every new comment to the discovery quote this example it's for the such we poll did come up makes a but hope to our web edition this obligation is starting background job with remote connection to our word doctor server server and where their doctor container is created way this exact version of Ruby with all that with the benchmarks you'd need for a week or for else depends on the recall and right after their benchmarks you'd is finished of their result is reported back to the web app where we can see updated shock here
is that the URI of revenge and there is so you can't use that benchmark on on the on the side bar on the left and on the right you will see the chart for these benchmark you can zoom and you can check out the comments and so on so no I have
some information about that architecture of from image and obviously we have 3 components all those all you can find all the 3 components and you have a there is a web application is equal with Dr. files and there is a report with the benchmarks used why do we use Dr. here because doctor is a great way to is a late environments to easily dependences and otherwise it would be just message to keep all those strong through this Beeld hindsight 1 error BN for RBM user and we have benchmarks you to reap along with that a lot of sneaker benchmarks for rails are for a b and then we update that they might you sometimes but now and draw like hundred maybe I maybe less of benchmarks me for benchmarks for itself and about from 10 to 20 benchmarks for us and we're working on adding new relevant marks no we benchmark at to record a TChain controller Action dispatched Ruta and use what about the bad that is just a simple rose 4 . 2 at run on your accrue the I where using delay job to run on a remote connection to our doctor server in background and we use high shots does library to render all those nice charts list of about results
of revenge and about real support there so now we have a set of benchmarks for else we're on deals for rails for every new version of rails we're working on support on per kilometer benchmarks and think quantity in in the next few days and what is exciting is that since the member there are already to 150 thousand The use for a bills of Meeker benchmarks and what's even more exciting is that there was already 1 performance regression of of there'll be caught and this performance regression could be sold on the next day after the comment was that after the wrong comment was made because we knew exactly which comment core of calls that this performance aggression what are our plants about this marking Ruby on Rails we want to implement the poet as benchmarks are for the U. Polat pastorales just report that the performance better than for the 1st author of we also want to make a weekly reports and in this case some contributors can get an e-mail about like in this attracted became 5 % faster or maybe not if their performance remain the same we simply don't need to send this e-mail and we also want to mention support because we already have a set of benchmarks for me and we can just from the same benchmarks against Jerry and so today we talked about what is the performance regression how they look like in rails how they can be sold how to find them how to track them what other tools without about building and benchmarks you'd you'd and how to maintain that taking all the performance but wait what if you're not errors can be a contributor and you're not planning to be ever how can all these traits help you to make your app faster and to avoid performance aggression and just vote performance questions right and to track your old at performance at 1st you can start by a by studying how do really primitives are basically the objects work and how to train them properly there is a great
book called Ruby under a microscope by special missing I mentioned this book in my talk because it covers all that that primitives how the the work not only in MRI but also in Robinia's in general if you're interested in but
more tricks like I showed about right fast through the you can check out the storm by Eric Michael sober MindScope right invested the and and
so that you can write a discourse like benchmark for your own rails at this
benchmark the concept of their applications that of the database is populated with some writers starts from the server in production environment and make a bunch of requests with up much bench for some other tools to some lists of and ones that are important for you like that in exchange for some of the page and then bring timings and run just from the event up after every comment on trellis and have some idea of the performance you can also see from subscribe to the
rails weekly mailing list but it's also called meeting rails and in these mailing lists of rows considered contributors right about what happened in the row 3 column this week what our new features that Fourier if we take some for regression in rail or fixed some issue stressor leans on my am about my dog so the first one is about their static on page we all those roles benchmarks are seems rails 3 . 2 and until rose 4 . 2 and another 1 is the initial prototype of the service to benchmark rails but I call it transfer 1st it's of open source and when you have and you can also check out the bench components on found in our organization I'm curious user names and Twitter and some please follow me and we also please don't hesitate to contact me if you're interested in this topic apartment about trails benchmarks about revenge and if you want to it is somehow so will be more than happy to discuss it out you can also check out their Twitter and website of the company and thank you so much for coming to this talk here of arms and move it to the from the ones in and you come
Softwarewartung
Vorlesung/Konferenz
Gebäude <Mathematik>
Lineares Funktional
Lineare Regression
Zwei
Versionsverwaltung
EDV-Beratung
Homepage
Gefangenendilemma
Software
Lineare Regression
Festspeicher
Projektive Ebene
Softwareentwickler
Regressionsanalyse
Aggregatzustand
Zentralisator
Bit
Punkt
Web log
Hochdruck
Gruppenkeim
Versionsverwaltung
Dicke
Mustersprache
Strebe
Betriebsmittelverwaltung
Regressionsanalyse
App <Programm>
Lineares Funktional
Statistik
Datentyp
Abfrage
Dateiformat
Kommutator <Quantentheorie>
Rechenschieber
Konstante
Rhombus <Mathematik>
Menge
Interpolation
Framework <Informatik>
Festspeicher
Dateiformat
URL
Zeichenkette
Fehlermeldung
Gefrieren
Zahlenbereich
Content <Internet>
Code
Framework <Informatik>
Variable
Datensatz
Authentifikation
Datentyp
Primitive <Informatik>
Inhalt <Mathematik>
Attributierte Grammatik
Linienelement
Objekt <Kategorie>
Regulärer Ausdruck
Zeichenkette
Patch <Software>
Attributierte Grammatik
Speicherbereinigung
Dämon <Informatik>
Zeitzone
Brennen <Datenverarbeitung>
Randverteilung
Subtraktion
Konfiguration <Informatik>
Hash-Algorithmus
Weg <Topologie>
Mathematisierung
Kartesische Koordinaten
Symboltabelle
Framework <Informatik>
Code
Datensatz
Prozess <Informatik>
Code
Hash-Algorithmus
Benchmark
Caching
Einfach zusammenhängender Raum
App <Programm>
Wald <Graphentheorie>
Kraft
Benchmark
Störungstheorie
Konfiguration <Informatik>
Dienst <Informatik>
Flächeninhalt
Festspeicher
Anpassung <Mathematik>
Codierung
Minimierung
Gruppenoperation
Mathematisierung
Versionsverwaltung
ROM <Informatik>
Computeranimation
Komponente <Software>
Datensatz
Algorithmus
Zusammenhängender Graph
Benchmark
Bildauflösung
Sichtenkonzept
Datenhaltung
Datenmodell
Benchmark
Hochdruck
Ereignishorizont
Gruppenoperation
Rechenschieber
Festspeicher
Mereologie
Gamecontroller
Speicherbereinigung
Programmierumgebung
Rechenschieber
Versionsverwaltung
Partielle Differentiation
Benchmark
Chatbot
Router
Solitärspiel
Twitter <Softwareplattform>
Machsches Prinzip
Partielle Differentiation
Systemaufruf
Iteration
Binder <Informatik>
Framework <Informatik>
Benchmark
Homepage
Resultante
Interpretierer
Linienelement
Datenhaltung
Zahlenbereich
Iteration
Benchmark
p-Block
E-Mail
Bitrate
Rechenbuch
Code
Computeranimation
Datensatz
Objekt <Kategorie>
Datensatz
Iteration
Körper <Physik>
Softwareschwachstelle
Code
Speicherbereinigung
Betriebsmittelverwaltung
Hilfesystem
Benchmark
Informationsmodellierung
Gamecontroller
Versionsverwaltung
Benchmark
Biprodukt
Programmierumgebung
Benchmark
Subtraktion
Statistik
Systemaufruf
Vorlesung/Konferenz
Keller <Informatik>
Videospiel
Menge
Reelle Zahl
Rechter Winkel
Versionsverwaltung
Benchmark
Programmierumgebung
Programmierumgebung
Benchmark
Resultante
Autorisierung
Explosion <Stochastik>
Punkt
Linienelement
Versionsverwaltung
Benchmark
Code
Ereignishorizont
Dienst <Informatik>
Twitter <Softwareplattform>
Softwareentwickler
Prototyping
Benchmark
Prototyping
Resultante
Wellenpaket
Hardware
Prozessautomation
Zwölf
Computeranimation
Intel
Dienst <Informatik>
Vorlesung/Konferenz
Streuungsdiagramm
Benchmark
Instantiierung
Hardware
Resultante
Einfach zusammenhängender Raum
Benutzerbeteiligung
RPC
Rechter Winkel
Prozess <Informatik>
Web-Applikation
Server
Versionsverwaltung
Vorlesung/Konferenz
Wort <Informatik>
Benchmark
Objekt <Kategorie>
Resultante
Abstimmung <Frequenz>
Weg <Topologie>
Gruppenoperation
Web-Applikation
Versionsverwaltung
Computeranimation
Reelle Zahl
Prozess <Informatik>
Lineare Regression
Programmbibliothek
Zusammenhängender Graph
Primitive <Informatik>
E-Mail
Bildgebendes Verfahren
Benchmark
Autorisierung
Einfach zusammenhängender Raum
Schreiben <Datenverarbeitung>
Objektverfolgung
App <Programm>
Lineare Regression
Ruby on Rails
Elektronische Publikation
Gebäude <Mathematik>
Systemaufruf
Benchmark
Mailing-Liste
Elektronische Publikation
Objekt <Kategorie>
Suite <Programmpaket>
Menge
Gamecontroller
Server
Computerarchitektur
Information
Programmierumgebung
Beobachtungsstudie
Prozessautomation
Message-Passing
Verkehrsinformation
Fehlermeldung
Mailing-Liste
Kommutativgesetz
Vorlesung/Konferenz
Primitive <Informatik>
Quantisierung <Physik>
Objekt <Kategorie>
Schreiben <Datenverarbeitung>
Weg <Topologie>
Datenhaltung
Mailing-Liste
Benchmark
Hochdruck
ROM <Informatik>
Ereignishorizont
Computeranimation
Homepage
Eins
Server
Programmierumgebung
Beobachtungsstudie
Benchmark
Objekt <Kategorie>
Sichtbarkeitsverfahren
Schreiben <Datenverarbeitung>
Web Site
Selbst organisierendes System
Weg <Topologie>
Open Source
Benchmark
Mailing-Liste
Computeranimation
Homepage
Datensatz
Dienst <Informatik>
Verbandstheorie
Twitter <Softwareplattform>
Lineare Regression
Zusammenhängender Graph
E-Mail
Beobachtungsstudie
Prototyping
Benchmark

Metadaten

Formale Metadaten

Titel Building RailsPerf, a toolkit to detect performance regressions in Ruby on Rails core
Serientitel RailsConf 2015
Teil 45
Anzahl der Teile 94
Autor Shatrov, Kir
Lizenz CC-Namensnennung - Weitergabe unter gleichen Bedingungen 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen und nicht-kommerziellen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen und das Werk bzw. diesen Inhalt auch in veränderter Form nur unter den Bedingungen dieser Lizenz weitergeben.
DOI 10.5446/30647
Herausgeber Confreaks, LLC
Erscheinungsjahr 2015
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract Performance regressions in edge Rails versions happen quite often, and are sometimes introduced even by experienced Core commiters. The Rails team doesn’t have any tools to get notified about this kind of regressions yet. This is why I’ve built RailsPerf, a regression detection tool for Rails. It resembles a continuous integration service in a way: it runs benchmarks after every commit in Rails repo to detect any performance regressions. I will speak about building a right set of benchmarks, isolating build environments, and I will also analyze some performance graphs for major Rails versions.

Ähnliche Filme

Loading...