We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Storing "Lots Of Small Files" in a Swift cluster

Formal Metadata

Title
Storing "Lots Of Small Files" in a Swift cluster
Title of Series
Number of Parts
561
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date2019
LanguageEnglish

Content Metadata

Subject Area
Genre
Abstract
OpenStack Swift is the massively scalable object storage of OpenStack. It can store billions of objects in a single cluster without any disturbances. But what happens if all of your objects are "small objects"? Because OpenStack Swift stores objects as files on XFS, disks will end-up storing millions of small files, creating large performance and stability issues. This is all related to the constraints of POSIX filesystems. During this presentation, we will give you an overview of these issues, and explain their root causes. You will then discover what we tried to overcome them, as well as the chosen solution. Eventually, we will present you results of our production deployment.