Optimizing folly using futex2
322 views
Skip to first unread message
andre....@collabora.com
May 11, 2021, 2:03:14 PM5/11/21
to Folly: the Facebook Open-source LibrarY
Hi there,
I'm the author of futex2[0], a WIP new set of Linux's syscalls that allows userspace to write efficient sync mechanisms. I would like to hear from folly's developers if the project would benefit from this new interface.
From folly's codebase, I can see that you are already familiar with futexes, but just in case:
* What's futex?
futex stands for "fast userspace mutexes". It's a syscall that basically exposes a wait queue, where userspace can ask to put threads to sleep and to wake them. The index of such queue is an unsigned int address (that is usually called a futex). It's value doesn't mean anything for the kernel, userspace is responsible to defining the rules of it and implementing the logic.
* Why futex2?
futex has long standing issues that seems impossible to be solved in the current interface, so we are designing a new set of syscalls taking those problems in account (more on that at [0]). It's not merged yet, and I'm researching some uses cases to validate and get feedback about the API. The new API has the following new features:
* Variable sized futexes (6, 16, 32 and 64 bits): we aren't restrict to only 32 bits anymore. From commit 146b07b5c "Extend futex to work with non-standard widths", it seems that this would benefit the project
* Wait for multiple futexes: this function enables userspace to wait on a list of futexes and wake on the first that triggers a wake. Our current use case for this is for game engines and to emulate WaitForMultipleObjects from WinAPI in Wine
* NUMA-awarenes: current futex has a single global hash table that is allocated in some node. Every futex() call that happens on the other nodes will have a access penalty. At futex2, there'll be a single table per node, so you can specify which one you want to operate on to increase locality.
The detailed description of the API can be seen in the documentation patch[1].
Thanks,
André
- futex2 talk at OpenSource Summit: https://www.youtube.com/watch?v=eG6GMBTcPQ8
[0] https://lore.kernel.org/lkml/20210427231248.22...@collabora.com/
[1] https://lore.kernel.org/lkml/20210427231248.22...@collabora.com/
I'm the author of futex2[0], a WIP new set of Linux's syscalls that allows userspace to write efficient sync mechanisms. I would like to hear from folly's developers if the project would benefit from this new interface.
From folly's codebase, I can see that you are already familiar with futexes, but just in case:
* What's futex?
futex stands for "fast userspace mutexes". It's a syscall that basically exposes a wait queue, where userspace can ask to put threads to sleep and to wake them. The index of such queue is an unsigned int address (that is usually called a futex). It's value doesn't mean anything for the kernel, userspace is responsible to defining the rules of it and implementing the logic.
* Why futex2?
futex has long standing issues that seems impossible to be solved in the current interface, so we are designing a new set of syscalls taking those problems in account (more on that at [0]). It's not merged yet, and I'm researching some uses cases to validate and get feedback about the API. The new API has the following new features:
* Variable sized futexes (6, 16, 32 and 64 bits): we aren't restrict to only 32 bits anymore. From commit 146b07b5c "Extend futex to work with non-standard widths", it seems that this would benefit the project
* Wait for multiple futexes: this function enables userspace to wait on a list of futexes and wake on the first that triggers a wake. Our current use case for this is for game engines and to emulate WaitForMultipleObjects from WinAPI in Wine
* NUMA-awarenes: current futex has a single global hash table that is allocated in some node. Every futex() call that happens on the other nodes will have a access penalty. At futex2, there'll be a single table per node, so you can specify which one you want to operate on to increase locality.
The detailed description of the API can be seen in the documentation patch[1].
Thanks,
André
- futex2 talk at OpenSource Summit: https://www.youtube.com/watch?v=eG6GMBTcPQ8
[0] https://lore.kernel.org/lkml/20210427231248.22...@collabora.com/
[1] https://lore.kernel.org/lkml/20210427231248.22...@collabora.com/
Reply all
Reply to author
Forward
0 new messages