Distributed Paper Stacks Meeting 1

[Alperen Keles]

Hi Everyone,

See you at 19.30(roughly in an hour) at Kampus Discord Channel(https://discord.gg/gQcQ83C7) where we hold our meetings to discuss Distributed Queues with Can!

Below is the abstract!

In this episode of Paper Stacks, Can Toraman will be discussing Achieving Task-Based Scheduling in Low-Latency Distributed Priority Queues.

In modern distributed systems, managing the queue efficiently is crucial for performance and responsiveness. Especially in multi-tenant environments, prioritizing different tasks (based on customer, task type, etc) is crucial. Yet, this area is still not highly explored, as many companies solve it by using different queues. Well, are more optimal ways, and how are they achievable in a distributed setting?

Distributed priority queues are still not highly explored in the academic environment, so will base our discussions on different theoretical and practical perspectives used in the industry:

• • Timestone, Netflix’s distributed queue used in its encoding service
  • ⁠B4, Google’s Wide Area Network to support requests from both GCP and Google applications.
  • ⁠Facebook’s FOQS to support horizontal scaling and multi-tenant environments
  • ⁠QPID: A Distributed Priority Queue with Item Locality

Links:

• • Timestone: https://netflixtechblog.com/timestone-netflixs-high-throughput-low-latency-priority-queueing-system-with-built-in-support-1abf249ba95f
  • B4: https://research.google/pubs/b4-and-after-managing-hierarchy-partitioning-and-asymmetry-for-availability-and-scale-in-googles-software-defined-wan/
  • FOQS: https://engineering.fb.com/2021/02/22/production-engineering/foqs-scaling-a-distributed-priority-queue/
  • QPID: https://ieeexplore.ieee.org/document/4725152

See you there!

Best,

Alperen