We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

High performance and scaling techniques in Golang using Go Assembly

Formal Metadata

Title
High performance and scaling techniques in Golang using Go Assembly
Title of Series
Number of Parts
611
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language
Production Year2017

Content Metadata

Subject Area
Genre
Abstract
In this presentation we present some optimizations we have developed for theMinio object server. To maximize hashing performances for bit-rot protectionwe have exploited the Go Assembly capabilities to natively optimizing BLAKE2and SHA2 techniques for both Intel and ARM platforms. In addition we willpresent a distributed locking package to aid with scaling a single serversolution into a multi server solution. In this presentation we would like to focus on a couple of developments thatwe have done for the Minio object server in terms of performance improvementsand scaling techniques. Regarding hashing we have developed Go packages for two hashing algorithms(BLAKE2 and SHA256) that take advantage of the Golang (or Plan9) assemblycapabilities. These packages exploit SIMD instructions of Intel and theCryptography Extensions for the ARM platform. The speed up range from 3 to 4times (Intel) to 100X (on ARM). Due to the use of Golang assembly there is noneed for a C-compiler in order to use these packages (when compiling fromscratch). For the Intel platform SSE, AVX, and AVX2 specific version areavailable and the most performant architecture is automatically chosen. In addition we have developed a distributed locking and syncing package forGo. Its main features are: \- Simple design for avoiding many tricky edgecases \- No need for a master node \- Resilient so other nodes are notaffected when a node goes down \- Drop-in replacement for sync.RWMutex-Automatically reconnect to (restarted) nodes \- Limited scalability of up to16 nodes We are successfully using this package for the Distributed version of theMinio Object Server