Logo TIB AV-Portal Logo TIB AV-Portal

Building a reasonably popular web application for the first time.

Video in TIB AV-Portal: Building a reasonably popular web application for the first time.

Formal Metadata

Building a reasonably popular web application for the first time.
Title of Series
Part Number
Number of Parts
CC Attribution - NonCommercial - ShareAlike 3.0 Unported:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal and non-commercial purpose as long as the work is attributed to the author in the manner specified by the author or licensor and the work or content is shared also in adapted form only under the conditions of this license.
Release Date

Content Metadata

Subject Area
Erik Näslund - Building a reasonably popular web application for the first time. These are the lessons learned when scaling a SaaS web application which grew much faster than any one us could have ever expected. - Log and monitor from day one. - Things will fail, be sure you know when they do. - Choose components which allow language interoperability. - Horizontally scalable everything. - Plan for database downtime. - Have a way to share settings between backend and frontend. - Have a way to enter maintenance mode. - And more... ----- My name is Erik Näslund - I’m the co-founder and Head of Engineering at Hotjar. I'd love to share the lessons learned when scaling a SaaS web application which grew much faster than any one us could have ever expected. Words like “big” and “popular” carry very little meaning, so let me define how big Hotjar is right now using some numbers. We onboard about 500 new users on a daily basis. We process around 250 000 API requests every minute. Our CDN delivers about 10 TB of data per day. We have roughly 3 TB of data in our primary data store (PostgreSQL), another 1 TB in our Elasticsearch cluster, and a LOT more on Amazon S3. These are the key things we wish we knew when we started. They would have made our life so much easier! - Log and monitor from day one. - Have a way to profile your API calls. - Things will fail, be sure you know when they do. - Have a way to keep secrets. - Everything needs a limit (even if it's really big). - Be wary of hitting data type limits. - Don't get too attached to a framework. - Choose components which allow language interoperability. - Horizontally scalable everything. - Plan for database downtime. - Features are a great way to test things out before launching them to the public. - Have a way to share settings between back end and front end. - Have a way to enter maintenance mode. - Require different quality of code for different parts of your application.
Computer animation Weight Right angle World Wide Web Consortium
Mobile app State of matter Multiplication sign Software developer 1 (number) Shared memory Data storage device Analytic set Set (mathematics) Product (business) Web 2.0 Type theory Computer animation Different (Kate Ryan album) Architecture Website Right angle Musical ensemble Procedural programming Game theory Position operator World Wide Web Consortium
Context awareness Code Length Multiplication sign Workstation <Musikinstrument> Mereology Expected value Computer configuration Physical system Exception handling Area Software developer Moment (mathematics) 3 (number) Bit Connected space Type theory Process (computing) Volumenvisualisierung Right angle Resultant Row (database) Point (geometry) Web page Histogram Service (economics) Link (knot theory) Data storage device Similarity (geometry) Field (computer science) 2 (number) Number Goodness of fit Arithmetic mean Profil (magazine) Form (programming) Standard deviation Matching (graph theory) Scaling (geometry) Information Interface (computing) Planning Database Line (geometry) Limit (category theory) Computer animation Integrated development environment Personal digital assistant Query language Speech synthesis Library (computing)
Point (geometry) Group action Overhead (computing) State of matter Code Multiplication sign Connectivity (graph theory) Source code Execution unit Motion capture Set (mathematics) Numbering scheme Similarity (geometry) Computer font Formal language Neuroinformatik Number Scalar field Cuboid Software framework Data buffer Physical system Form (programming) Electronic data processing Scaling (geometry) Inheritance (object-oriented programming) Concentric Wrapper (data mining) Shared memory Data storage device Database Limit (category theory) Fehlererkennung Human migration Type theory Process (computing) Computer animation Personal digital assistant Statement (computer science) Quicksort Table (information) Data type Library (computing)
NP-hard Axiom of choice Group action Context awareness Code Multiplication sign Source code Open set Parameter (computer programming) Mereology Computer programming Subset Exclusive or Roundness (object) Different (Kate Ryan album) Cuboid Office suite Endliche Modelltheorie Extension (kinesiology) Social class Physical system Scripting language Sound effect Bit Bulletin board system Content-addressable memory Type theory Skeleton (computer programming) Curvature Arithmetic mean Telecommunication Order (biology) Quicksort Resultant Asynchronous Transfer Mode Wide area network Web page Point (geometry) Server (computing) Implementation Service (economics) Overhead (computing) Data storage device Dreizehn Graph coloring 2 (number) Product (business) Goodness of fit Term (mathematics) Natural number Ring (mathematics) Ideal (ethics) Energy level Software testing Utility software Traffic reporting Form (programming) Condition number Content (media) Database Line (geometry) Software maintenance Limit (category theory) Geometric quantization Computer animation Integrated development environment Personal digital assistant Universe (mathematics) Formal grammar Statement (computer science) Wireless LAN
Computer animation Series (mathematics) Physical system
people 3 halves and find tell you the ball will billion reasonable popular work for son you know if it is thank and 1st of all I can't get becomes confirmation to the rights of the this without my notes
and this is it goes wrong so I can talk about building recent problem where the occasion for the 1st time goes by the lucky enough into being able to on state building design something that grew quite quickly and they got to learn to be scaled Howie quicker than I would be expected to do so along a lot of this during this time and like share what you learned soulfully you can do at least in doing the mistakes we and make your own unique ones so
who might work has and I'm the cofounder and chief architect at a company called popular what shot both the name of the company and product is a set of web analytics and the old so basically this means a lot of data ingestion we you installed on almost 200 thousand sites in the world right now so a lot of data coming in but it is not later so my development along the and 86 6 wrote my 1st game it was in that was some problem with respect but I thought it was so I'm talking programming and I've been and also that that position between different text types and throughout the years but start Python about 7 years ago now and this 1 I get something like most so so since I'm going to talk to you about something recent popular recently debate that let me some defended the definition of what I think we need big rights so what you right now process around 400 thousand maybe requested many the procedure and it is about 10 to about something that the users and we have roughly 3 to most data from a data store the consequence and the tutor right ElasticSearch cluster ensemble the 5 fold terrabytes animism so much our definition of recently popular reason for today
still use reason standard solutions so takes that isn't anything out of the ordinary is here and makes memcached McAliskey Python and as such to post this it works uh and basing the wealthiest runner followed Michael was workers in this case believe it or not and at some point we will or start using all funds in using and you can put all these things it's probably the great match for us but for now the plane process-based Michael whiskey skills so now we have some context let me you start out with what the large during the last 2 years so know
moment the day 1 this is something that we've messed up a bit because we only started building and and it once we start at the top at that point and we had so much no data coming in so let's spend quite a lot time cleaning things up the before lectures easily to stop logging in aggregating log from day 1 and you know keep it looks like after the problems the other was unanimous cleaning up when we need to understand the like it's kind of that not mentioned looks have weighted profile you read like so we use when using a scale out a smaller and it's great and love it like 95 per cent of the time but every now and then you have a slightly in innocent line of Python code that causes some really related and have the profile both cold and database queries is great we have to consider that Our ourselves only connection and question profile equals 1 to any of and a guy called the cost instead of the normal results that makes standpoint and return the sea profile data and this development profile and having no doubt he's way to get approval data from like the a called in the lab environment in just a few seconds such a great it makes a profile a lot more and you get a much better understanding of the system as a whole so highly recommended to have a way to add following queries from like environments you sometimes in Python code that takes time sometimes it's in the database but you'd be surprised how often light the Python code is actually you know it was still a little mistake muscular me that's really haven't processes no 1 thinks straight so at some point we had at some point oxygen from the quite works but you know some background yeah they feel ongoing without because it was thought failure it's it's a for some unknown reason it didn't throw an exception thing like that because the world is many things that but it just the size so it's just as important to know when things are not happening as you know is when you know about the Japanese so it's all this by adding simple concept of your basic stations and your results a job expectation is something simple like I expect histogram every don't results is simply log entry from a job that rights when it's some based just have a status and point that's cool but some of the parts service and checks that all expectations are satisfied all the time that way we know the jobs run and the time and run successfully so we whereabouts safeguarding against things that fail explicitly and things that fail silently just as important and and also a 3rd party systems to monitor your insistence on your monitoring face have we keep secret what Europe has Everything else started out as an experiment such as you learned to diligent about not maybe keeping the external API piece is often forms of in hindsight about of you know it's then as a development team we realize OK maybe the best idea that everyone has access to all the corpus systems like 3 guys you know and love environments so I recommend you something like and some people similar like from day 1 it's going to be of this is didn't so it's time we like had to start you know keeping secrets we change all the API that stuff so have a way to keep secrets from day 1 the this indicates what everything is limit even if so a good example areas we have constant of parts we can basically time according its use for the envisioned it be used for people OK this recording the user visited pages recorded library users used slightly different some that tagged each recording with Euphrates coming from the port systems like the little so that meant some users ended up with 400 thousand different types we show that knowledge of my speech to problem 400 thousand each its electron on options not render work or interface broke terrible because within the limits in its user 4 very creative and if you give them away put like limitless amounts of information they will and these limits goes because you like it goes the ice link the fields stuff like that most obviously goes databases length the never never ever allow unlimited perfect of but unlimited is about if you give users way to put on a limited amount of data in system they will eventually it took like yeah but number of things the of on here is slightly more interesting of safe and much more
surprising PostgreSQL a data store use In important prevent for us in the form for the idea we eventually that limits for so we had our to something billion rose the that was can take take trying to solve that when everything was and because I didn't understand it never worked with data the scale before but it happened so think about when trying to design a scheme try to think ahead a year or 2 and what sort of try is a possibility that could end up but like reading datatype limits for use this type if you think you even those since the beginning of it's not expensive it's just not be full 72 conscious choice but think about how your data will grow and if possible put mentoring place this as well when you're about to reach the limits false the 1 and the same time like planned migration don't get too attached a framework right now we're using Frost and foster scalar and the risk of false source and it works pretty well we should promptly what but at for 500 thousand wheat was prominent it's starting to have like a significant overhead because most of our requests are like really quickly process so the framework matters this of course you know it depends on the use case but us it matters the of only will probably going have a transition to something else so a good advice to minimize the pain of doing that is to use framework agnostic libraries as much as possible flight is still out is a great example because you know it works like it has about the 4 basic everything and if it's on east you 1 is over I don't have anything against using were like called the rappers like foster skill this is basic doesn't do that much it's just myself but if you were to you know if you which way from frost you can easily implement what fosters scale conductor so thin wrappers fine otherwise I try to avoid and framenet-specific libraries it's kind of like you know the in framework in this flexibility she's components which allow for language and Robert so were different from a nearby John but we have about but profits and 1 % recall based in luxury for performance reasons running inside and Unix within the state of saying human system called on Cuba initially great system boxes and I can only and this caused some issues when we based just want to look old put some simple things in you that end up being a much bigger thing now because you know we can't conclude that is biconnected so when possible choose components component libraries servitor level in or for that for greater language interoperability mixed so you see if you have like performance-critical but to just take it out and there is something of time for data based on time the other in the beginning all of our database migration migrations were simple because we had no users and what and its order and at some point in time we ended up you know you could just do a basic computation table statements and anymore because they taking significant amount of time fair enough throw some detailed makes it neutral alleviate some of the but at some point you have to like introduced kind of downtime however this nice think that helps and try to be coupled data ingestion from data processing as much as possible I need we do it capture data from user put you precisely the way you become much more resilient to having data based on the unit just for you minutes you need take it down little change but if you have like this queuing here buffer it's not always possible to this obviously but is a great thing to do is when you can and find way to share settings between back and front code we introduced a couple of still a bunch of times simply because we believe we call things from back front and then we change 1 of have the front and back and couldn't agree that values so this is just a silly and you get and as a very simple solution we ended up having of things the Jason from which contains a share settings it's injected using engine exercise and that's where I think we Jason and the font and can be you know the concentrations so super simple watersheds if things go there and the number of this kind the shared settings a good Dr. getting cold like we don't things like error codes and stuff called basic knowledge said something similar problem have a way
to go and it is not what I mean is mode is basically a little page saying remember story it's not a nice when you have to bring it up but this probably happened to 1 of us at some point and that is a credential having and we basically have very little switch to turn on and off the make mistakes and when doing the maintenance stage be careful and let's have a few weeks external dependence this is possible because you know you probably want on like when you database of aggression so don't store the switch to content in the database because of that and that was the 1st from that it does not also on a maintenance page we've code in the class communication where people can talk with a support group it's really good idea I think to keep communications open with users and when bad things happen and feature flats greatly testing the performance and everyone so at this point and I was talking with bacon wouldn't 1 lecture release things weren't too sure about to everyone so we introduce feature flights we have both server-side and client-side features like so basically is this part of the right requires this feature and this part of the API request the the way we can do that we can do beta testing with a limited group of people and the we can also do things like enabling things depending on which type of use results say if you the program get this feature so that that versatile tool to have if they start thinking in terms of 1 of 3 features which is highly recommend very simple to implement great things in the toolbox accept different quality of coal from different parts of the system this is person looking for formality because he was developed you kind of get taxed your what you and want to support in every but it don't because then you run out of time and so for example we require all our user-facing color to be properly tested performing well all these things imagine you have like back office report for internal use it's OK if it performs also if it takes 5 seconds to generate but think about things these things up from before starting to build a new feature and hi good doesn't documentation be how well this the the form how well this need to test In in an ideal world everything would be properly documented tested and perform but when you need to prioritize think about a lot from it helps a lot and the but basically the most noteworthy things we've looked that kind of not unique but surprising things that of and I'm sure we still have many new things the last but this is from thank
you for listening and if we want to be each of the why was there this will contain chosen what 1 of the reasons and how to hold model useful sort of give you a skeleton instead of just use general room yeah OK yeah well it's still happening started out with a different order the and but for some of of our and the performance things we good you know we don't want to go right was the other so use and without the actually is allowed to me allows to drop on like a mid-level still you would really very Wireless and the jungle where and is even I prefer I can say like this month temperature level in the jungle or is awesome but they're still out when you really need these we have performed stations and use the is specific it features and stuff I found it a bit could have done it general however we already decided that because simple benchmarking class quite a lot faster and and even without the universe without will less and what not so wouldn't really have a natural flight into John we forget what I mean so and then this gluttony was a good choice and I think there's something thanks and worsens to this thanks but did you get the more details on the implementation of year the maintenance more page yes so we're having to do that act culturally yes absolutely and it's a very simple thing that the 3rd 2nd background of hard problems were basically postings the bucket we have the service bulletin of the themselves so and in maintenance mode is basically an rounded Groningen script but through Jenkins and basic check box maintenance not instead so is always the service we got this takes about 20 30 seconds and and what they basically do in during a built by the line executed ideas we actually have conditionals there its context and basically this is simplest like if maintenance mode sure this page that it's the the and so if I were to lots and then into the effects of and of course and there are endless infected guidelines but to the world what the the kind of thing that has to be useful especially for our company and that would be uh writing utilities for testing the server just a small lines because you can right to new that about you that the use the prepared environment uh not very production the native so you can just quick around your appliances production and that's what this is about the source of goods and and I think and I can get reporting deployable only like what we call up so you can easily just a bullets use and to make it a bit to very fast and social anchored to the quantization here so that the various called of the profiling so do you use anything else other than having the ability to see the light profiling only a lot of things that I think this because I think it's I haven't seen that much before but we had the users of and we use peaks that statements in Portuguese it's all something very very small extension that's extremely low overhead less than 1 % in most cases and basically generalizes queries so dependent and independent of where parameters it like groups quest for you and it's like mean exclusion time-averaged you standard stuff and so if you want like reading find so where's pages that statement for day-to-day monitoring neural and that's basically for performance monitoring and how do you leave me that profiling the molecule you of users of that symbol and
it's said you have to be looking in the system there's normally use of but then we have like a little it's series like for certain uses the so this is the and right and that greater called requires cities so in anymore also during lunch things are going