We're sorry but this page doesn't work properly without JavaScript enabled. Please enable it to continue.
Feedback

Ceph RGW Message Queue API for Serverless Computing

Formal Metadata

Title
Ceph RGW Message Queue API for Serverless Computing
Title of Series
Number of Parts
637
Author
License
CC Attribution 2.0 Belgium:
You are free to use, adapt and copy, distribute and transmit the work or content in adapted or unchanged form for any legal purpose as long as the work is attributed to the author in the manner specified by the author or licensor.
Identifiers
Publisher
Release Date
Language

Content Metadata

Subject Area
Genre
Abstract
A proposal to support AWS SQS API natively in Ceph RGW, for advanced Serverless computing use cases. Bucket notification has become widespread in many applications including AI/ML and Edge Computing. We have demonstrated that Ceph RGW bucket notifications trigger Serverless functions including Knative and AWS Lambda. Such workflows can be characterized as event push and event driven. We’ve identified yet another workflow that allows Serverless computing frameworks to be more preemptive in auto scaling. The workflow is based on the event queue polling model, the opposite of event push, that allows Serverless computing frameworks, such as KEDA, to preemptively scale the functions based on the queue length. Such workflow is more advantageous and lightweighted in dealing with highly bursty event traffic and reducing Serverless function cold start overhead. In order to support this workflow, we propose a new set of message queue APIs, modeled after AWS SQS, for Ceph RGW. In this talk, we’ll present the overview and planning of this technology to the community.