We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Speed up the monolith

Formal Metadata

Title
Speed up the monolith
Subtitle
building a smart reverse proxy in Go
Title of Series
Number of Parts
490
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date2020
LanguageEnglish

Content Metadata

Subject Area
Genre
Abstract
GitLab is a ruby on rails application, but this didn’t prevent us from having fun with Go. Learn how we decomposed our monolith by writing a smart reverse proxy in Go that handles I/O intensive operations. A technique that every web app can use, regardless of the company stack. We set a deadline for releasing a cloud-native version of GitLab and put a team of engineers to work planning the helm charts, splitting several components into independently scalable PODs. The team faced a few challenges. GitLab’s main codebase is written in Ruby, which has a global interpreter lock. We relied on NFS to asynchronously upload files from our workers fleet. Removing shared file system by uploading directly from the controller was not an option. We wanted to move to an object storage based solution, but that was a paid feature and we had to port it to the open-source codebase. Oh, we also needed to make sure the rest of our engineers could keep shipping new features at our regular monthly cadence. At the same time, we were planning our infrastructure migration from Azure to Google Cloud. Removing this intermediate state, where a file is on GitLab server NFS but not yet uploaded to the object storage, would have made the migration a lot easier. We had to remove the NFS dependency to make GitLab easily deployable on Kubernetes and we needed a performant multi-cloud object storage uploader viable also for on-prem installations, a solution that would work for a single server setup up to Ggitlab.com scale. Luckily we already had written workhorse, a smart reverse proxy written in Go for handling git operations. It was time to extend workhorse capabilities leveraging the full power of goroutines. We had a plan, but the devil is in the detail. Allow me to guide you through this journey. During the talk I’ll tell you how a ruby-on-rails company began to write Go code, how we implemented an object storage uploader inside our proxy, the problems we faced, and tradeoffs we took to deliver this in time.