Profiling PHP applications

Video thumbnail (Frame 0) Video thumbnail (Frame 1016) Video thumbnail (Frame 1916) Video thumbnail (Frame 2823) Video thumbnail (Frame 4458) Video thumbnail (Frame 6085) Video thumbnail (Frame 6951) Video thumbnail (Frame 8675) Video thumbnail (Frame 9776) Video thumbnail (Frame 11439) Video thumbnail (Frame 13320) Video thumbnail (Frame 14517) Video thumbnail (Frame 16321) Video thumbnail (Frame 18172) Video thumbnail (Frame 19198) Video thumbnail (Frame 20943) Video thumbnail (Frame 21909) Video thumbnail (Frame 23050) Video thumbnail (Frame 23948) Video thumbnail (Frame 26142) Video thumbnail (Frame 30114) Video thumbnail (Frame 31849) Video thumbnail (Frame 35194) Video thumbnail (Frame 36170) Video thumbnail (Frame 37697) Video thumbnail (Frame 39127) Video thumbnail (Frame 40008) Video thumbnail (Frame 41030) Video thumbnail (Frame 43123) Video thumbnail (Frame 44091) Video thumbnail (Frame 48152) Video thumbnail (Frame 49269) Video thumbnail (Frame 51252) Video thumbnail (Frame 52638) Video thumbnail (Frame 56300) Video thumbnail (Frame 58539) Video thumbnail (Frame 66003) Video thumbnail (Frame 67646)
Video in TIB AV-Portal: Profiling PHP applications

Formal Metadata

Profiling PHP applications
Title of Series
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Release Date

Content Metadata

Subject Area
Help my client is complaining some parts of the application are slow. Now what? Profiling! What is profiling and how can we measure the performance of our application? There are several tools we can use. Once we have the tools, how do we approach profiling. What to look for. And caveats to avoid when profiling. To finish up, we should avoid that our client is complaining about performance, how can we pro actively use profiling to improve our application. When finished we will end up with a nice toolbox of profiling tools and good ideas how to do profiling and avoid some common mistakes that might distract you from the real optimisation.
Cartesian coordinate system Regular graph
Run time (program lifecycle phase) Cartesian coordinate system Reduction of order
Operations research Software developer Operator (mathematics) Software developer Core dump Website Bit Client (computing) Cartesian coordinate system Twitter Twitter
Run time (program lifecycle phase) Information Characteristic polynomial Mathematical analysis Mathematical analysis Cartesian coordinate system Mereology Measurement Process (computing) Semiconductor memory Profil (magazine) Process (computing) Mathematical optimization
Area Email Shooting method Information Profil (magazine) Cartesian coordinate system 2 (number) Number
Dependent and independent variables Run time (program lifecycle phase) Multiplication sign Computer network Database Cartesian coordinate system 2 (number) Befehlsprozessor Software Causality Read-only memory Semiconductor memory Befehlsprozessor MiniDisc MiniDisc
Server (computing) Information Musical ensemble Cartesian coordinate system
Cache (computing) Computer file Information Profil (magazine) Ferry Corsten Configuration space Bit Client (computing) Extension (kinesiology) Timestamp
Complex (psychology) Functional (mathematics) Information Ferry Corsten Maxima and minima Cartesian coordinate system Limit (category theory) System call Number Cache (computing) Crash (computing) Visualization (computer graphics) Logic gate
Revision control Overhead (computing) Profil (magazine) Ferry Corsten Debugger Bit Resultant
Scripting language Slide rule Functional (mathematics) Information Open source Computer file View (database) Bit Price index Proof theory Profil (magazine) Configuration space Representation (politics) Configuration space Extension (kinesiology) Computing platform Library (computing)
Graph (mathematics) Service (economics) Profil (magazine) Weight System call
Demon Web page Functional (mathematics) Demon Information Cartesian coordinate system System call Web browser Proof theory Telecommunication Profil (magazine) Function (mathematics) Configuration space Quicksort Extension (kinesiology) Identity management Computing platform Extension (kinesiology)
Axiom of choice Profil (magazine) Computing platform
Demon Web page Demon Information Cartesian coordinate system Web browser Power (physics) Revision control Telecommunication Profil (magazine) Telecommunication Endliche Modelltheorie Extension (kinesiology) Computing platform Extension (kinesiology)
Befehlsprozessor Software Information Semiconductor memory Ferry Corsten Multiplication sign Cartesian coordinate system Computing platform Local ring
Web page Default (computer science) Slide rule Presentation of a group Computer file Information 1 (number) Bit Library catalog Cartesian coordinate system Product (business) Cache (computing) Semiconductor memory MiniDisc Website Social class Tunis
Point (geometry) Presentation of a group Sequel Demo (music) Code Profil (magazine) Real number Maxima and minima Video game Client (computing) Number
Demo (music) Interactive television Zoom lens Cartesian coordinate system Product (business) Computer configuration Read-only memory Function (mathematics) Revision control Quicksort Electronic visual display Message passing Force
Uniform resource locator User profile Graph (mathematics) Read-only memory Profil (magazine) Function (mathematics) Befehlsprozessor Counting Mereology System call
Point (geometry) Multiplication sign Website Code
Demo (music) Computing platform
Uniform resource locator User profile System call Inheritance (object-oriented programming) Read-only memory Profil (magazine) Function (mathematics) Befehlsprozessor Zoom lens Counting 2 (number)
Inheritance (object-oriented programming) Service (economics) Read-only memory Function (mathematics) Befehlsprozessor Data storage device Information Counting Mereology 2 (number)
Service (economics) Query language Demo (music) Combinational logic Table (information) Row (database) Physical system
Service (economics) Backup Algorithm Server (computing) Computer file Sequel Multiplication sign Combinational logic Code Cartesian coordinate system Power (physics) Subject indexing Graphical user interface Response time (technology) Semiconductor memory Query language MiniDisc Website output MiniDisc Table (information)
Befehlsprozessor Dependent and independent variables Befehlsprozessor Robot Electronic mailing list Website Block (periodic table)
Database normalization Server (computing) Integrated development environment Information Demo (music) Mereology Cartesian coordinate system Physical system
Web page Point (geometry) Sign (mathematics) Uniform resource locator Default (computer science) Graph (mathematics) Home page System call Form (programming) Product (business) 2 (number) Default (computer science)
Web page Point (geometry) Server (computing) Information Structural load Online help Bit Stress (mechanics) Cartesian coordinate system Mereology Product (business) 2 (number) Software bug Cache (computing) Integrated development environment Profil (magazine) System programming Video game Musical ensemble CAPTCHA Physical system
Operations research Complex (psychology) Overhead (computing) Graph (mathematics) Sequel Information Software developer Multiplication sign Set (mathematics) Maxima and minima Continuous integration Mereology Cartesian coordinate system Normal operator Product (business) Twitter Pointer (computer programming) Integrated development environment Website Musical ensemble
Canonical ensemble Point cloud
our next speaker will be like the
boulder is one of the regulars in the organized and he's going to talk about something I think we all breath and that's namely profiling PHP application so like so yeah profiling PHP applications first can you hear me in the back yes okay that's good so first
who if you like slow applications yeah you can leave now we need we need chairs is it only about faster execution time
or would we like to use less resources
or is it only as much request as we can
get and usually what happens is we start profiling when we see this so everything
is on fire your client is on the phone yeah my site is down and everything is slow and it breaks and yeah what what's going on you don't know you have no clue so that's why we are going to profile our application but first hi I'm Mike he
said it already so short introduction I'm on Twitter you can send me things on Twitter whatever I do some development a little bit of operations at Studio Emma we do Magento and pimp core if you want
to know something about it find me afterwards that's it so first profiling
profiling is the process where we find the runtime characteristics and measure the performance and I'll do the performance analysis for optimization of our application is that clear so what information do we
get from this profile where is something happening where is something slow where is something using a lot of memory such things what is well we want to know what is causing this but what is using it that's the first thing we know we also learn how much that's already a very interesting part of it but then you have
that question why my application is fast yeah you see if I click it it's fast yeah but do you have substantial information to confirm that your application is fast and what is fast for you so define fast is 2 seconds fast for you then yeah okay fine don't profile but can you handle a burst of traffic so your marketing department has a has a great idea and they don't tell you they will send a great marketing campaign and then it happens bomb everything down and then you ask around yeah did somebody do something oh yeah we sent their mail to all our customers yeah thanks so yeah optimizing your application has benefits in all areas you can imagine
there you get this guy I don't need a profiler and know what slow are you sure you're kind of shooting in the dark if you are yeah you probably can make an educated guess where something is slow but if you don't have the numbers for it it's still a guess so the points of
interest we are going to look at while we are profiling our execution time so
this is the overall thing we get a response in two seconds or maybe we like one second better but we also get the CPU time spent on your application and the memory usage also interesting to have is how much disk IO does my application cause because this could really help you if you get a burst or something what we also have is network time do we lose a lot of time asking using external api's or same same with external delays like is the response from your database slow or did you just kill your Redis in some magic way that is suddenly super slow or other databases you are trying to use
so now how can we profile our application with PHP who would do it
with PHP just something in PHP magically and we'll get something no one yeah maybe not so you can get some information you get the time and if you do requests you can even get it from your server server superglobal but yeah you get actually nothing because you've got it you get it already from another [Music] another way so first XD back who knows
you who of you knows that you can profile with XD back oh that's already quite a lot so exci burke is an extension you know it's for PHP usually we use it for debugging but it also provides us with profiling it needs a
little bit of configuration to let me to to make it allow you to profile and you need an external tool to analyze what it creates so it creates a cache client file and then you can visualize it with some external tool so what I usually use is enable the profiler write it somewhere and cache grind out percentage you that's just a timestamp you could be smarter about this and get more information you can find it in a documentation of exit back and one once you get those cache client files you can open it in a tool like Q cache grind or
gate crash grind or whatever blah blah blah cache grind to there is I think there is a web-based one also then you get some information but it's not super easy to to understand what's going on
you can get some visualization but yeah there is not really something showing up as being very slow and there is one larger drawback with these tools that they have a maximum there is a limit on the number of function calls it will show you so usually you can get it up to 500 function calls but this comes from Magento this is a very complex PHP application which has a lot more than 500 function calls so you will probably not see all the information you want probably what you need but it's not very easy so it's hard also exit back is not
really a profiler but it's a debugger with profiling capabilities so it also introduces kind of a large overhead it became better with newer versions but still it might the result might be a bit off then I guess the people who knew
exit back also knew XH broth but maybe there are other people's will also know that XH broth is out there so XH broth well these days with PHP 7 we don't no
longer use it as XS proof itself but it continued in the Tideway is PHP extension there is another another open-source extension that has a similar capabilities but personally I used the Tideway extension it's also some a little bit of configuration needed and you use well in my personal preference X hu e which is an external tool to visualize the profiles so it's very simple the only thing that's really important here is that you do the tight waist o to prevent library on 0 because that's for the tight waist sauce platform you also include some prepend where you actually initialize your profiling and it's not all there but eventually if you get in the shutdown function you collect your information that it gathers from your application and write it somewhere so that was the wrong way so then you have
X hu e which is empty in this in the beginning and once what I also forgot with XZ Burke is that everything you do with PHP is profile so you do not have to do something extra to have a command line script profile because if it's on its own if you do command line if you do request whatever you enable it it's always on that's for me it's very interesting so then you get some runs in it you get an overview and in the details you already get some information this slide contains something that might be interesting because it already has an indication of a problem so at the end of representation if I didn't go go deeper into composer autoload include file ask me about it what's the problem you get your function information annual
so can get a call graph which might be overwhelming but it's very interesting so the for me the important thing here
is that it's still a local set up so I don't need to act any external services and it profiles everything doesn't look too fancy but it has most well all the features you want to to get some some information out of it you also have tied weights itself which
is an sas that you can also configure so
it's also they're the same thing you get an extension you configure it it needs daemon on your PHP application to send the information to the platform the the interesting thing with tight ways is that it has a 30-day trial that it offers you all its features so you really can evaluate what it can do it looks rather nice it also has a p.m.
functionality that is out of the scope of this presentation and if you want to profile something you use an extension and so it the profiling is on the amount of you you say I encountered some sort of problem on this page I now want to get a profile out of it so you say proof take take profile and then title is will will reload this page and give you a profile in their dashboard gives you some information about function calls when what happens and so on and so on so
it's more than a profiler it's fairly easy to use but yeah it's trial to paying this really depends on if you already have have an AP in place and you only need to profile locally or you really new into this and you can benefit from a platform that has everything so that's the kind of a choice you have to make and then we also
have black fire which is very similar and set up to Tideway so it's again the
same it's an extension it's similar it uses a demon to give to get the communication done between your application and their platform the biggest drawback for me is that it's a freemium model where they're free features are not really useful to evaluate what the power of the platform is so you actually should start with the paying version where you see everything and then decide if you want to continue with it so it's exactly the same practically
like with tide ways you click a button and then it profiles a page don't you get some information I only have screenshots from the free version so
it's very easy to use but the drawback is in the freemium you only have the wall time and a memory usage if you want the CPU time if you want see IO if you want to see Network if you want to see all the other things you really need to start paying from day one that's what you choose so that's for me the biggest drawback it does not not offer you all the features to evaluate if you want to continue with this platform my biggest
suggestion here is pick one and learn it once you learn the one you have picked you're probably will encounter things you don't like about it so you don't find your way properly in it or it doesn't offer sufficient information on something specific you would like to know and maybe another pot platform offers it then it's time to start looking around for start shopping for you could say but first you should dive into one to really know its quirks and how it's gonna behave on your applications that it's especially interesting if you already have in-depth knowledge about your application you will quickly learn what is missing in the profile or that you are using to summarize the the tools it's like eggsy berg xh probe our local set up sideways and black fire our SAS solutions so if you really like the solutions go for one if you want to keep it local use XD back or exit probe but
now we still need some idea how we are going to approach the profiling itself so what are we going to look for what
are we looking for first we look for slow things for example yeah for example the contact page on a Magento website if you configure it default it's probably the slowest page on your site weird isn't it also what you really want to optimize for is popular things so if you're an e-commerce probably your catalog pages will be hit a lot more than product page so if you really want to profile to gain well to be more future-proof you are going to watch at a popular thing so you can preemptively optimize them before someone trashes your site and application initialization that's the one we saw at the the earlier slide where composer autoload include file is really slow if there is only one takeaway from this presentation remember if you see this composer at Lloyds include file that is really slow just think op cache memory is too too small this is not obvious but yeah we search for it a few hours to find out that actually what happened was oh composer wants to add a new a new class so it includes the file but the op cache memory is too small so yeah the one that was already in well many others were in but the oldest ones were pushed out but they were used again a little bit later so it continuously accesses the disk to get the information instead of its class definitions coming from up cache this was a not obvious issue
so and then you go and analyze a profile first follow the hotspot so usually it's marked as the more red the slower it is so usually if you see the red fading away that's kind of the the point you're going to start watch and yeah what's happening here does it do a lot of my sequel calls or does it so does this method something really weird like I saw an example where I don't remember where I saw it but it was another presentation where I was like yeah we have an HTTP client and it does something really weird and far far away it did something like I calculate pi to 20,000 numbers behind the comma yeah if you want it if you want to do that but yeah that's weird so it can be weird code so fix that try again if it's better you're good so now a little demo from something
we did in real life first I'm going to start with a more simplified example
is this readable or do I have to zoom it a bit better zoom yeah oh it's good yeah so there is some background to it yeah that's the one so the story here is we had to create a application where they said well we're not gonna use euros we're gonna let our users by some sort of coins and not not crypto just you give 60 euros and get one thingy and our products will all be based on yeah let's say we take a base value of 60 euros and everything will be based on that so usually something we will sell will be one two or three coins yeah okay so something was built and it was like yeah someone wants to buy coins so we started with yeah the coin value it's like let's take 100 euros and yeah we want to be generous and we're gonna give them to other people so we'll buy ten oh that's too fast yeah sorry so no we want to buy in euros so we want to be generous so we want to have ten coins to distribute to other people and that's
pretty okay so initially if we're going to look at this so we get here one run and it takes
one second so we get into details keep in mind this is a very oversimplified example to show you an easy part into profile so what I usually do is yeah I go into
call graph and then it's always too big
and now there is just a cheat in it where the sleep is now taking most of the time so it's this is the reference it takes one second so our codes we are using is okay at
some point in time there was another department in that company that said no we're not gonna do that but the whole idea of their website was based on the coins so then they said yeah let's change that we're gonna use coins of one year oh and go wild so yeah with a thousand it's probably okay it's not
really that much doesn't really matter and then they said yeah but we have a
lot of our own personnel and we want to give them a budget of 2,000 years or something for courses on our own platform so I'm going to buy for 100,000 euros and then distribute him across the company that's when we started to see weird things because the platform was completely not built to have 100,000 coins ordered at once
so it's becoming slow so we know the reference is one second now we're at 13 seconds to get the ID we started looking
at things in the profile the zoom is working against me yeah that that didn't
go well yeah that's better
so what we come to see here is we follow the hot spot and yeah it's probably too small now so we follow the hot part and instead of crawling I want you to zoom thank you yes and it's like yeah store coins what the hell is this thing doing it takes 10 seconds probably should make it better
so if we look at it it's based on an actual it did happen so demo service
coin service yeah this is actually very readable so what happened is there was the ID of yeah we go to the slow one so we store things and what happened is yeah just at one coin and the table contains one coin and per row so we start to insert 100,000 coins and go wild in a for each and then do other stupid things like try to check how many unspent coins there are so the combination of these two is really disastrous so because we didn't originally create our system to support really small values it escalated quickly and here you can see you can easily
follow this paths to find out where things go wrong and start refactoring here because we do not actually want to insert 100,000 rows we want to we don't really want to do 100,000 queries we really want to do one query to get all these things in and it will be a lot faster so but this is a very obvious example but we'll get to others later
first we have some popular bottlenecks so I already said that slow coat really
weird algorithms or spending too much time on things you don't really want to spend on that sequel queries what you also get is in the beginning your site might be very fast without indexes on your sequel on your tables eventually you might see slow down so use your indexes but don't overuse them because that might bring you to other problems what we also see a lot is disk i/o usually triggering your application to be slow and usually it's a combination of something externally is doing too much disk i/o on your server your PHP wants to do a lot of disk i/o and suddenly things go bad so what could happen is certainly a backup kicks in and reads a lot of files without Io limiting or whatever and it causes your application to go slow what we also already saw is usually we have an application and it has a cron somewhere and the chrome goes wild because yeah someone did not tell you they were going to do some marketing campaign or update everything at once and once you see swapping because you have memory issues on your server you have to think of you have to think about other problems other things too also if you want to be fast you need to keep in mind that not everyone wants to be fast so what we also encountered in the past was like yeah we have we have an API yeah do do whatever what you want with it but they found it acceptable they found it acceptable to have a response time of one second but if you want to go fast yeah you have to do something about it and then it
happens everything goes down and it breaks and there they are the angry customers so the first thing you have to
do is don't panic if your site goes down just relax it's not gonna kill not gonna get any better if you're panicking too so first just throw in more CPUs if it's needed throw in more RAM and then see to block probably some mouses actors so you probably find some bad BOTS list and block them and when things start to
normalize then try to find out what happened and try to find out how you're gonna improve it then find the slow parts start profiling and get on with it
so for your production environment it might be interesting for everyone to have your monitoring have an application performance monitoring on it what I find personally very interesting is to have a top on it I don't know how many people know a top yeah but if you have it on your server you can when everything is back to normal you can replicate what happened see when it started to explode and get some really useful information to avoid these kind of issues in the future and if you really need to you can go and s trace things if you really want to go deep so I have one other demo where it's not exactly obvious that something is slow
this will be there so this is Magento if there are no there are no products in it so it's yeah it's empty but doesn't matter it's gonna show up that something is slow I already said it it's the the contact page so if we go to contact it's gonna be around 1 point something or 2 seconds and if you go look at it we see like the default was 380 milliseconds and the contact page did 2.7 seconds for a page with only formed it's weird but if we go into that yeah there is a lot of stuff going on here
this call graph will take a while because and it's gonna be intimidating
[Music] whoops and then it was gone oh it's so small now yeah so it's back yeah but you as you can see it there is a lot of red everywhere and yeah where to start you will need some inside knowledge of your application well you will eventually follow the hot part again and we'll find out something peculiar I think it's there somewhere no yeah let's not try to find in life it doesn't help but somewhere if you get into this problem you will find out somewhere that it renders your page in PHP again that's kind of an expected behavior because you thought yeah well I have all my caches on so you don't have to rerender this page it's somewhere in cache if you dig a little bit further you will find out it has a Magento CAPTCHA and it which is rendered server-side surprised you cannot catch this page so it slows down a lot but for these kind of problems you also already need a lot of application knowledge all well inside knowledge about the application you are using to find out actually what is going on so you will eventually recognize a problem oh it renders the HTML so this is not what I expected so why does it do it and eventually you find out it tries it also it always renders a CAPTCHA you don't want it to do so you need to find another solution
so in closing find your slower things for me my preferred go-to solution is xh probe and xh ue because we use an APM and production so we get some starting points from there and then we only use profiling locally to really fix the problem also don't trust profiles from your application performance monitoring for example when your system is under stress so if you have a high load on your server that that information will not be useful will not be correct also what you can take away from this is make your popular pages cacheable it will help you a lot and if you can profile periodically in your development environment - if you have some reference you will know that your application slowed down while adding new features or for example you had to do a bug fix and it's suddenly one and a half seconds instead of one second maybe you have to reconsider the fix are there questions
yes so the question was have you ever tried New Relic yes we use that as an APM in our production environments but that was out of the scope of exactly profiling so that's that's also the reason my preferred go-to solution for profiling to improve things is xhu II with x8 profit x8 Yui because we get some pointers from New Relic where things are probably slow and if you can reproduce this locally we can get ahead and so yeah you get all you get you already get useful information from New Relic so you can reduce your search time in the cold graph where you very already know from New Relic yeah okay it's about my sequel queries it's its elastic search that's that's responding slow it's something else so yeah it's useful you have other competitors of New Relic I think the auto doc is one and tight ways sideways also offers you a minimal set of APM so yeah it's it's very useful to have it the only drawback we've seen lately is that in really complex applications New Relic might have too much overhead so we we switch it on and off like if we expect a problem we turn it back on but if we want normal operation we turn it off because we noticed at some sites between 15 and 30 percent overhead which is really big yeah yeah the more complex your application becomes the bigger the penalties of New Relic even though they are actually a tool that you want to run in production that you want to have in production to get this information but yeah we lately seen it if your application is really big it might become no longer really beneficial to have it yeah for example yeah other questions yes yes so the question was is there a possibility to get this in continuous integration I summarized it correctly yes it's possible so that I didn't mention is like Blackfyre has this that you can make it profile during your continuous integration so you really see the trend if your application is slowing down slowing down over time so you can make it a part of your continuous integration it it might be very useful also what I did not mention is there is a more newer tool which is called PHP spy but I only tried it briefly and I'm not sure if it's really that easy to use for request based profiling but it works very well for command line tools to profile those and it gives you a nice fire graph which is very easy to interpret to read and to understand but it's fairly new so I have to dig into that deeper more questions no well thank you and you can give me feedback on joined in or on Twitter or whatever [Applause]
[Music] whoa [Music] you are in real-time preview at the minute