Merken

Easy App Metrics

Zitierlink des Filmsegments
Embed Code

Automatisierte Medienanalyse

Beta
Erkannte Entitäten
Sprachtranskript
what the hell out of the of the of the of the of the of whole thing of the time and the and the other is that about that is that the regulatory that right so the of so as many of you know Django came from a journalism background and we've had some talks today that relate Django in like literary terms and so on a take it back to the journalism roots and do a little bit of the who what where when how of you AP metrics but I'm going to screw up you were a little better so 1st who we and the founder of evolution systems and had been what we're trying for a really long time but for this talk the most important of source world is of a project called Jango act metrics that helps makes up doing some of
this a little easier and will get to that toward the end but that we're going to talk about most
importantly the I when I 1st proposed this talk I thought you know that the shorter time frame because covering how did metrics psycinfo 45 minutes with right images do really boring and repetitive but I would actually do the landscape is moving things have gone a little easier and so I came 25 minutes with on how to do metrics the you know the code part that has become really really easy and the hard part is what should be tracking and why should I be tracking so let's let's cover some of those things
and I come from cancers I mention of a lot of Genghis from Lawrence Kansas by state has some trouble with of believing facts and so on but I know you guys I am driven by fax you can show me some numbers and I believe that you somewhat accurately collected those numbers are much more likely to take action then you know what I feel like that but should be a little bigger or I feel like we should make this feature more so this helps give you ammunition with your co-workers in your bosses for things that you want it done or what you don't wanna have to do so obviously we collect a lot of metrics and have pretty graphs for the the kinds of things you expect ops you know what's the load on disk space those kinds things so we need we need for for that
answer but we can also use metrics to decide on development prioritization if you have a and also to be able to tell customers no but you will get requests from your coworkers for customer saying you know it be really great if you could make it do this I know you do you handle CSV but really really really want an actual Excel download you're not too keen on even though working you know what you're the only customer uses that feature at all right maybe we should prioritize that because it's just not something that's being heavily used we also use
metrics to make design decisions so all designers are gonna hate me for this slide and I did like everything long right let which is funny because this is the slide that me the longest actually design right so that the desired have gotten so easy to do it right that you have to work hard to do it wrong but oppression might come does it allow you to use Comic Sans anymore and so but you you there's a lot of things in our sights on on web apps where we do them kind of out of of history we keep with the controls at the top but with the controls again at the bottom where you track where they actually use it may be that the nature of your app is the user always close to the bottom of the page because that's where the new estate is and so they're only using the controls at the bottom and so get rid of them at the top you can get some screen real-estate back maybe other control should go up there at the track the difference not just I used feature X but I used feature x from this spot on the page that that become important from a piece of information we can use that against the data with with internal things internal politics who should get more funding who should get a new staff member what should we bother to have those meetings what we talk about it on the world about you getting rid of the walls between that than arts but we can use that to get rid of some walls between that of the OPS in sales and marketing because if I can give the sales guys some numbers that they didn't know about marketing guys some insight into what they're warning now there my friend and they had a little more understanding when I tell them hey this is going to take 6 weeks to build a you sure we really wanna do this I can do that right now or whatever else so this can be a way to get a break down some walls inside your organization the the so we're glad lots of crazy metrics but we have to remember that you know correlation is not causation necessarily so if you see a big spike in your graph that doesn't necessarily mean that the other thing that happened at that same time point is what caused it right so it's usually just a signal of I need to dig deeper into this problem so what kinds of should we be tracking
specifically so most of you can probably come up with the so before I can without any to cover costs but about casts on metrics there's a couple of things to consider the the actual storage cost right i'm a collect all this data how much is it going to cost me to store this the time it takes me to implement it on and then there's also there what this does to the runtime right I don't wanna slow my app down considerably by tracking 9 million things on every page on an every action so I need balance them but keep in mind that metrics are pretty darn cheap and if you set your retention policies right with graphite where we hosted services will give some of that you can store a metric for like 20 minutes of so of space for a 10 year period so don't be too afraid of collecting too much data but do keep in mind that the you know if they're not absolutely free do we know the track the deep things right low disk space and memory usage how many with processes we have going on many worker processes are running most everybody knows to track that stuff and it's a good place to start right it's better than nothing on but there's other things you might not think about they're good available to overlay and look at along with feature rollouts involves that can help you so have of real problems so table sizes right not just how much disk space is the database used what happened did this deployed all the sudden this tables blown up and size was that that make sense or is that maybe something really bizarre above that we haven't detected here and how long the book coming employs we doing how long take you know how many we have marriages but how many times a somebody else is eating into the server that can be really good metric of without doing enough parts automation and and some and you can easily script up and keep track of this more poorly than just keeping track of some raw numbers we need to make sure that we segregate some of this data so like I user's bottom example here support tickets we don't want just to get greater result but we want you know is this created by somebody internal versus external but department it we will have a little bit more dimension to some of these these data points so that we can see or the support tickets are generated by customers or they are generated by that 1 guy and that 1 department who's really just kind of you know a pain Frank but so we have basic act metrics and this is kind like that marketing and sales once the seeing growth factors right you would see like sign-ups and free paid upgrades downgrades without of that kind of stuff on how many people like you on Instagram or whatever but but the list of other things like what e-mails are we sending the customers by type we we know how many we send probably if you use male gonna service like they give you an invoice that he said 1812 last year last week but about what type is all of those all password resets we up on your honey as people logging in logging has anyone ever log out you know that can be a design decision that we don't spend any time I was lot of page looks like because 1 person does it every 5 days of so you must have the cookie just times are not actually physically worn out the so having that data helps us to see how things are being used in the AP and where we should be focusing our top right and bug fixes and new features in performance you know a lot of people track how many items are in the queue but they don't necessarily track how long individual types of jobs take in that particular q so you know how long does it take from I click password reset to I've told male gonna send e-mail if that's a 2nd goal it that's 15 minutes knuckle right because I'm a user I think something's wrong and now I'm hitting 5 or 6 times and I'm putting up your q because I kind of expect to get an email within a minute or so you know I don't necessarily expected to be instantaneous but I expected to be a minute but that's a cache hit rates you know were using caching but are you know are we using it effectively economic API usage is this article on the left public API access is this all you know also just a normal customer also our customers what and point to being hit the most sometimes you'll find a feature is used very heavily from an API and not use it all from your desktop apps or use the lot desktop and API but never on mobile and so you can de-emphasize that in design in terms of benefits and we should be tracked random internal stuff when we have meetings how many staff members that we have then is mother correlate but sticks to how many people were working I'm having and occasionally we might see that when 1 particular staff member leaves then blogs go up ago that maybe that person is more important than we think that job was more important and we think of how many times your co-workers have having with more than 40 hours and you know are we generating problems because of burnout we can track the sex and that that we can't correlate them without some sort of visualization and you know we should check chat messages and emerging is used and you know what we we can have some fun with the right I I I had this coffee example here of did you do it in the opposite to the start box and was and event a whatever did did not so much that I want you to track your coffee consumption at that level but to think about that enter in terms of metrics right it's not just how many celery tested we do it's which of them and why and what you know what kind of spectrum were we doing this was this army free users use that feature versus paid users the the so in the will you should be doing that you shouldn't yesterday and but we can talk a little bit about time travel here a little bit of how to do this but you need it before you need to have some metrics in place before you really need them right so start collecting them early even if you don't have a good plan to visualize so how do we go about doing this right so this the easy stuff you know Google Analytics everybody's got that on the site right and and then there's no support some stuff that helps with op eaten was well as sponsors here use knew relic data dogged those are pretty easy to set up and they give you a lot insight into your apps of iconic paid some of these services because before they existed everybody had become directs us for performance and now they have these services and they can see how it's as 1 query that's making my side slow and and so this other services like Lombardo and paneling keen diode that let you kind of create your own metrics and create ways to graft them and it's the kinds of things that you would do if he were hosting your own metrics out without having to do any of the OPS worker set up yourself than those do they have different pricing and some different features echoes out but if you really cannot get to the the top level right you got you gotta do it yourself so how do you do it yourself the best way to visualize is were far yeah the other despots systems out there but if you don't have 1 picked out that you really love you should look at the of it is beautiful you can show it to bosses and clients and everybody likes looking at at nobody's gonna say this looks like matplotlib or something that you know we we did for fun but but you know so pretty is important if people don't like looking at at they're not going to look at it on a regular basis and refined action besides being beautiful it's easy to use it can target graphite and invokes the B for its sources and Mongolia can target multiple of them so you can kind of Charlotte your metrics as you need to to grow so you can have some metrics pulling from cluster 1 and some from cluster to all on the same dashboard
so you need it at the mistakes I see people make is they have 1 big dashboard with 9 thousand things on and nobody's delicate look at that and was affine and useful so create special-purpose dashboards for the kinds of things that you're looking at what things do we care about in this release what things the sales care about what things marketing care about the but once you've got them set up your look at them a lot maybe Stickum on a big screen and lobby and have a rotating kind of stop looking at them but you need to be able to the beginning and look for new and interesting facts so spend some time a read out again just overlaying different metrics to see see what falls out the and the main reason people don't do metrics is because graphite is a pain in the butt to install who here has wasted a day trying to get at the graphite set up a character that this is an so mostly made of maybe people don't use it because it's not so there is an easier way and that's graphite API it gets rid of all the hard parts of the set up and it's just the part you need to work with so check out graphite PPI and then influx the is the
new kid on the block it's written go in it's the hot new thing on its main difference is you can have individual metric points and then attach arbitrary tags to them so you instead of having in graphite you would have CPU load for US east of you know the box 5 With that involves the DVDs have CPU load and have your availability zone and an instance type B just tags against that and so you can do more aggregations inquiry more easily so how do you transport these metrics around you use that the that these really easy service to run the biggest mistake people make is they have us that's the system and they have like 20 service all talking to it and and that actually gets slow and can slow your wrapped up so 1 1 on every spot you're going to collect metrics and used it to show them to wherever they end up getting store another thing that people don't tend this notice is that if you're using lab session a taking logs right near doing stuff with logs and showing them off into a service you can use log stash to generate metrics off of logs based on regular expressions so you don't assert to retool you to put this stuff and you can generate lot metrics off of walls and if you want to find metrics then all you need is a little API that you can just soc these n from your DOS script and show them in the rest of your your back so I needed time travel most of these systems you can punch you can push into with a specific time so when did this happen so you can go back and replace all those large even storing S 3 to come up with what the data you're looking for water to show trends over time so how do we do this with my the stats the update is very simple you discuss the client and you have a counter that you can increment you can increment and decrement increment by more than 1 Our municipal samples you can have gages so right now my stress levels at 25 per cent of 100 and is at an arbitrary numbers here in the you timing how long did I sleep last nite when I got about 8 hours use the distorted milliseconds you can have timings of how long this particular thing take and then invokes DB vicious that very similar but that they take a specific format that's the measurement name whatever arbitrary tags you 1 0 attached to that and then the time value and then what the value of the field was so during Django 10 out of 10 and I'm attaching that was in Austin barbecue was a predominant food so to the
Django bet J. U. of metrics it is a low utility that is kind of like a haystack in that it abstracts out where your storing this so you can use Django our metrics and target makes panel vibrato stats the store them in your database the inredis knows to give a personal homepage 1 o'clock semantics on you don't wanna set all that stuff up to storm in the database and you can do some simple aggregations and roll up of were inredis so you can put this in when your app is small and change where your storing them over time without having to retool every so you can have a simple metrics like the code examples so that just increments the code example metric or we can time things with the context well we can set a Gauge check if the NSA knows more about what your users are doing with your application and you do you're doing something wrong you up so like and I think that you know if you using is Google Analytics they probably know more about what is going on in your at you so embrace as agent and in then Bill mn questions without
for 8 things for denies review so
my question would be how how you decide how to generate the data if you're going to do it directly from your application or you're gonna log 8 and then have something portioned out like hold you will hold you make the call and of what comes into the decision I I think that the biggest there is
do NOT have the code written out by got a big gap that's got a lot of things going on probably going to try to collect the metrics from the outside some of them to try and pull off of logs of the other reason that I might use of orthologs instead of inside the ad is because of performance if I don't need the answers in real time or near real time but trying to collect that metric any other way would slowly up down on a push it into elastic search combine and pole or or use log stash and pull those out and maybe we translate that into something like graph I just so that I can have an overlaid with the other governments thanks do you have support for the influx TV tagging kind of stuff that metrics I don't yet I hope to have time to add that end up for for today but I haven't done invokes the back in because they want to help with that and we also but the rest of that I was wondering um few any tools recommendations for more like more those esoterica I metrics you're suggesting like the the culturally type 1 also answer yeah I mean Walpole requests and you can still do Hubble do Web books right so you can listen for those in Engish of metrics and you can go and backfill that pretty easily by adding the API things like do we have a meeting you know like I you maybe after calendar year book for this spring meeting and automatically pull them in some maybe a very small little webapp had a button and say hey I'm in a meeting where you know maybe you want to add a good meeting verses that meeting like you know I I know that when I was able to you know show that all the requests were coming when I was IT manager although most of like 70 per cent of my request reading of most of my time was coming from 1 department when that department complained about how long things were taking I said well you know almost all my requests are coming from you and you you know how that you give me some of your budget so I can hire more people and we can both flow better and these I absolutely I didn't realize that you're doing all the work right but if I didn't have numbers like that I just be you know he said she said kind of arguments of logical not your fault it's the state's fault the hi from of the text of the mention it and a sort of almost culturel question and give any tips on making your co-workers Oracle co-developers area care more about performance so I think that anything you know I if you measure it people care about right right you can show them that your work over the last week slowed everything down by 10 per cent they're gonna start to do is check that it may be valid like it there is that there is no way to make it any faster and the feature was necessary and so you you can get in the beginning it too hard on them for taking longer with more functionality but you know Moses and nobody is noticing no one's looking and the if if you're not seeing it on a change by change kind of basis you you can figure out it's harder to figure out the root cause if you only look at your performance numbers every 6 months are only when the starts complain that upon pages taken too long to load by what was it I'd have the will go look to the last 8 thousand commits and will try and figure out what were the problems where or who the problem that's right it could be that 90 % your developers are kind of performance savvy and there's the 10 % that art and they just need cement or but if you actually looking at this stuff on a regular basis you know somebody gets an e-mail and it's like with the testing and I have projects where if testing falls below 80 % coverage it doesn't pass EI and then can be deployed so boom everybody's just gonna keep that up and stroke sometimes to this being the numbers but I give it something maybe you need performance testing like the guy I was talking about the last talk and have that be something that's stopping the build thanks so this is kind of 2 in question on graph finer you can get data from a lot of different places and you put in a lot of different places but I knowing what you to put together it actually get meaningful insights is fairly kind a pain in the arse end is out something worker phonics pennies you take these disparate of pieces of data and Turner matching together and do it makes it easier yes it makes it easier I it's not easy that's why I say getting should go play you know a little bit and nobody has all week to sit around and just you know graph numbers on top of each other and see what sticks but you know try and not to try and look at everything every now and again and see does any of this match up with anything else doesn't know those overworked mean more bugs no it's so quantity and that last question the you touched on earlier on your talk about how these are for free but I was kind a curious if there were any like good guidelines are any hard and fast rules in terms of monitoring every little thing the office or in your project whatever I mean I think that you know and we can all agree that if you need the higher a person to track shoe size at your company probably not a metric were collecting rights like there's time costs actual costs you know so I it's about the act but if you have a time tracking system already that you're using for other reasons than polling data out of that and show that it into something that you can graph against other things you know that's partly with the 30 minutes it takes to do that and if you've got a calendar were you know all meetings happen in the conference room and you have of an map that tracks when's the conference room and use it may be worth collecting meetings you know and it may be worth collecting employee happiness if you have other reasons there during the HR is gonna system that surveys everybody every 3 months it might be worth collecting and shoving you know min max and average mean inter in the graph on and see if anything shakes out thank you on at the time thank you Frank until it past my my my 1st all
Bit
Softwaremetrie
Rechter Winkel
Evolute
Besprechung/Interview
Projektive Ebene
Wurzel <Mathematik>
Physikalisches System
Quellcode
Term
Systemprogrammierung
Softwaremetrie
Rahmenproblem
Rechter Winkel
Mereologie
Rotationsfläche
Bildgebendes Verfahren
Code
Computeranimation
Nichtlinearer Operator
Softwaremetrie
Last
Benutzerschnittstellenverwaltungssystem
Mini-Disc
Gruppenoperation
Zahlenbereich
Ungerichteter Graph
Softwareentwickler
Raum-Zeit
Aggregatzustand
Stellenring
Freeware
Twitter <Softwareplattform>
Raum-Zeit
Homepage
Freeware
Client
Last
Statistische Analyse
Kontrollstruktur
Skript <Programm>
Punkt
E-Mail
Korrelationsfunktion
Caching
App <Programm>
Datentyp
Kontrolltheorie
Bitrate
Ereignishorizont
Entscheidungstheorie
Dienst <Informatik>
Verbandstheorie
Rechter Winkel
Benutzerschnittstellenverwaltungssystem
Login
Festspeicher
Zahlenbereich
Server
Tabelle <Informatik>
Subtraktion
Selbst organisierendes System
Automatische Handlungsplanung
Datenhaltung
Weg <Topologie>
Zufallszahlen
Speicherbereichsnetzwerk
Datennetz
Datentyp
Videospiel
Mobiles Internet
Google Analytics
Raum-Zeit
Default
Rechenzeit
Indexberechnung
Programmfehler
Summengleichung
Office-Paket
Caching
Cookie <Internet>
Resultante
Korrelationsfunktion
Bit
Prozess <Physik>
Punkt
Web log
Natürliche Zahl
Web-Applikation
Übergang
Prozess <Informatik>
Minimum
Visualisierung
Nichtlinearer Operator
Prozess <Informatik>
Datenhaltung
Abfrage
Speicher <Informatik>
Quellcode
Frequenz
Teilbarkeit
Rechenschieber
Softwaremetrie
Funktion <Mathematik>
Information
Schwebung
Message-Passing
Web Site
Quader
Ausnahmebehandlung
App <Programm>
Hausdorff-Dimension
Stab
Gruppenoperation
Zahlenbereich
E-Mail
Punktspektrum
Term
Overlay-Netz
ROM <Informatik>
Message-Passing
Multiplikation
Softwaremetrie
Mini-Disc
Passwort
Warteschlange
Speicher <Informatik>
Touchscreen
Implementierung
Streuungsdiagramm
Rechenzeit
Bildauflösung
Mailing-Liste
Physikalisches System
Zeitreise
Quick-Sort
Endogene Variable
Warteschlange
Mereologie
Basisvektor
Mehrrechnersystem
Mini-Disc
Stab
Term
Punkt
Parser
Login
Übergang
Client
Statistische Analyse
Skript <Programm>
Einflussgröße
Overlay-Netz
Prozess <Informatik>
Speicher <Informatik>
Applet
p-Block
Instantiierung
Zeitzone
Web log
Dienst <Informatik>
Generator <Informatik>
Softwaremetrie
Datenfeld
Twitter <Softwareplattform>
Dateiformat
p-Block
Normalspannung
Instantiierung
Subtraktion
Quader
Wasserdampftafel
Stapelverarbeitung
Zahlenbereich
Regulärer Ausdruck
Gebäude <Mathematik>
Zentraleinheit
Datenhaltung
Quellcode
Softwaremetrie
Datentyp
Stichprobenumfang
Fokalpunkt
Skript <Programm>
Speicher <Informatik>
Touchscreen
Logarithmus
Statistische Analyse
Physikalisches System
Zeitreise
Regulärer Ausdruck
Last
Mereologie
Visualisierung
App <Programm>
Konvexe Hülle
App <Programm>
Abstraktionsebene
Datenhaltung
Mathematisierung
Besprechung/Interview
Softwarewerkzeug
Kartesische Koordinaten
Kontextbezogenes System
Nonstandard-Analysis
Code
Formale Semantik
Homepage
Softwaremetrie
Softwaremetrie
Speicherabzug
Eichtheorie
Gleitendes Mittel
Speicher <Informatik>
Innerer Punkt
Quelle <Physik>
Bit
Extrempunkt
Web-Applikation
Kartesische Koordinaten
Fastring
Login
Lie-Gruppe
Homepage
Eins
Mixed Reality
Wurzel <Mathematik>
E-Mail
Softwaretest
Parametersystem
Lineares Funktional
Physikalischer Effekt
Homologie
Entscheidungstheorie
Arithmetisches Mittel
Softwaremetrie
Verbandstheorie
Rechter Winkel
Projektive Ebene
Message-Passing
Aggregatzustand
Lesen <Datenverarbeitung>
Subtraktion
Baum <Mathematik>
Mathematisierung
Besprechung/Interview
Zahlenbereich
Term
Code
Weg <Topologie>
Benutzerbeteiligung
Mittelwert
Datentyp
Softwareentwickler
Graph
Matching <Graphentheorie>
Schlussregel
Physikalisches System
Quick-Sort
Programmfehler
Mapping <Computergraphik>
Echtzeitsystem
Flächeninhalt
Last
Basisvektor
Identitätsverwaltung

Metadaten

Formale Metadaten

Titel Easy App Metrics
Serientitel DjangoCon US 2015
Teil 32
Anzahl der Teile 46
Autor Wiles, Frank
Mitwirkende Confreaks, LLC
Lizenz CC-Namensnennung 3.0 Unported:
Sie dürfen das Werk bzw. den Inhalt zu jedem legalen Zweck nutzen, verändern und in unveränderter oder veränderter Form vervielfältigen, verbreiten und öffentlich zugänglich machen, sofern Sie den Namen des Autors/Rechteinhabers in der von ihm festgelegten Weise nennen.
DOI 10.5446/32762
Herausgeber DjangoCon US
Erscheinungsjahr 2015
Sprache Englisch

Inhaltliche Metadaten

Fachgebiet Informatik
Abstract What to collect How super easy it can be Techniques for collecting today, but using tomorrow Processing your metrics out of band to not slow down your code Tips for scaling in large systems Visualizing your metrics with Graphana

Ähnliche Filme

Loading...