[QUEUED scylla next-4.4] dist: tune fs.aio-max-nr based on the number of cpus

[Commit Bot]

From: Takuya ASADA <sy...@scylladb.com>
Committer: Avi Kivity <a...@scylladb.com>
Branch: next-4.4

dist: tune fs.aio-max-nr based on the number of cpus

Current aio-max-nr is set up statically to 1048576 in
/etc/sysctl.d/99-scylla-aio.conf.
This is sufficient for most use cases, but falls short on larger machines
such as i3en.24xlarge on AWS that has 96 vCPUs.

We need to tune the parameter based on the number of cpus, instead of
static setting.

Fixes #8133

Signed-off-by: Takuya ASADA <sy...@scylladb.com>

Closes #8188

(cherry picked from commit d0297c599afadd808f00f88ac4b9f55631cb543a)

---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
--- a/dist/common/scripts/scylla_prepare
+++ b/dist/common/scripts/scylla_prepare
@@ -28,6 +28,7 @@ import distro

from scylla_util import *
from subprocess import run
+from multiprocessing import cpu_count

def get_mode_cpuset(nic, mode):
mode_cpu_mask = run('/opt/scylladb/scripts/perftune.py --tune net --nic {} --mode {} --get-cpu-mask-quiet'.format(nic, mode), shell=True, check=True, capture_output=True, encoding='utf-8').stdout.strip()
@@ -99,6 +100,14 @@ def verify_cpu():
print('\nIf this is a virtual machine, please update its CPU feature configuration or upgrade to a newer hypervisor.')
sys.exit(1)

+def configure_aio_slots():
+ with open('/proc/sys/fs/aio-max-nr') as f:
+ aio_max_nr = int(f.read())
+ required_aio_slots = cpu_count() * 11026
+ if aio_max_nr < required_aio_slots:
+ with open('/proc/sys/fs/aio-max-nr', 'w') as f:
+ f.write(str(required_aio_slots))
+
if __name__ == '__main__':
verify_cpu()

@@ -113,6 +122,8 @@ if __name__ == '__main__':
os.remove('/etc/scylla/ami_disabled')
sys.exit(1)

+ configure_aio_slots()
+
if mode == 'virtio':
tap = cfg.get('TAP')
user = cfg.get('USER')
diff --git a/dist/common/sysctl.d/99-scylla-aio.conf b/dist/common/sysctl.d/99-scylla-aio.conf
--- a/dist/common/sysctl.d/99-scylla-aio.conf
+++ b/dist/common/sysctl.d/99-scylla-aio.conf
@@ -1,2 +0,0 @@
-# Raise max AIO events
-fs.aio-max-nr = 1048576
diff --git a/dist/debian/debian/scylla-kernel-conf.postinst b/dist/debian/debian/scylla-kernel-conf.postinst
--- a/dist/debian/debian/scylla-kernel-conf.postinst
+++ b/dist/debian/debian/scylla-kernel-conf.postinst
@@ -9,7 +9,6 @@ if [[ $KVER =~ 3\.13\.0\-([0-9]+)-generic ]]; then
else
# expect failures in virtualized environments
sysctl -p/usr/lib/sysctl.d/99-scylla-sched.conf || :
- sysctl -p/usr/lib/sysctl.d/99-scylla-aio.conf || :
sysctl -p/usr/lib/sysctl.d/99-scylla-vm.conf || :
sysctl -p/usr/lib/sysctl.d/99-scylla-inotify.conf || :
fi
diff --git a/dist/redhat/scylla.spec b/dist/redhat/scylla.spec
--- a/dist/redhat/scylla.spec
+++ b/dist/redhat/scylla.spec
@@ -205,7 +205,6 @@ if Scylla is the main application on your server and you wish to optimize its la
# We cannot use the sysctl_apply rpm macro because it is not present in 7.0
# following is a "manual" expansion
/usr/lib/systemd/systemd-sysctl 99-scylla-sched.conf >/dev/null 2>&1 || :
-/usr/lib/systemd/systemd-sysctl 99-scylla-aio.conf >/dev/null 2>&1 || :
/usr/lib/systemd/systemd-sysctl 99-scylla-vm.conf >/dev/null 2>&1 || :
/usr/lib/systemd/systemd-sysctl 99-scylla-inotify.conf >/dev/null 2>&1 || :

[Commit Bot]

From: Takuya ASADA <sy...@scylladb.com>
Committer: Avi Kivity <a...@scylladb.com>

Branch: next-4.3

dist: tune fs.aio-max-nr based on the number of cpus

Current aio-max-nr is set up statically to 1048576 in
/etc/sysctl.d/99-scylla-aio.conf.
This is sufficient for most use cases, but falls short on larger machines
such as i3en.24xlarge on AWS that has 96 vCPUs.

We need to tune the parameter based on the number of cpus, instead of
static setting.

Fixes #8133

Signed-off-by: Takuya ASADA <sy...@scylladb.com>

Closes #8188

(cherry picked from commit d0297c599afadd808f00f88ac4b9f55631cb543a)

---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
--- a/dist/common/scripts/scylla_prepare
+++ b/dist/common/scripts/scylla_prepare

@@ -27,6 +27,7 @@ import platform

import distro

from scylla_util import *

+from multiprocessing import cpu_count

def get_mode_cpuset(nic, mode):

mode_cpu_mask = out('/opt/scylladb/scripts/perftune.py --tune net --nic {} --mode {} --get-cpu-mask-quiet'.format(nic, mode))
@@ -97,6 +98,14 @@ def verify_cpu():

print('\nIf this is a virtual machine, please update its CPU feature configuration or upgrade to a newer hypervisor.')
sys.exit(1)

+def configure_aio_slots():
+ with open('/proc/sys/fs/aio-max-nr') as f:
+ aio_max_nr = int(f.read())
+ required_aio_slots = cpu_count() * 11026
+ if aio_max_nr < required_aio_slots:
+ with open('/proc/sys/fs/aio-max-nr', 'w') as f:
+ f.write(str(required_aio_slots))
+
if __name__ == '__main__':
verify_cpu()

@@ -114,6 +123,8 @@ if __name__ == '__main__':

@@ -200,7 +200,6 @@ if Scylla is the main application on your server and you wish to optimize its la

[Commit Bot]

From: Takuya ASADA <sy...@scylladb.com>
Committer: Avi Kivity <a...@scylladb.com>

Branch: branch-4.3

[Commit Bot]

From: Takuya ASADA <sy...@scylladb.com>
Committer: Avi Kivity <a...@scylladb.com>

Branch: branch-4.4

dist: tune fs.aio-max-nr based on the number of cpus

Current aio-max-nr is set up statically to 1048576 in
/etc/sysctl.d/99-scylla-aio.conf.
This is sufficient for most use cases, but falls short on larger machines
such as i3en.24xlarge on AWS that has 96 vCPUs.

We need to tune the parameter based on the number of cpus, instead of
static setting.

Fixes #8133

Signed-off-by: Takuya ASADA <sy...@scylladb.com>

Closes #8188

(cherry picked from commit d0297c599afadd808f00f88ac4b9f55631cb543a)

---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
--- a/dist/common/scripts/scylla_prepare
+++ b/dist/common/scripts/scylla_prepare

@@ -28,6 +28,7 @@ import distro

from scylla_util import *
from subprocess import run

+from multiprocessing import cpu_count

def get_mode_cpuset(nic, mode):

mode_cpu_mask = run('/opt/scylladb/scripts/perftune.py --tune net --nic {} --mode {} --get-cpu-mask-quiet'.format(nic, mode), shell=True, check=True, capture_output=True, encoding='utf-8').stdout.strip()

@@ -99,6 +100,14 @@ def verify_cpu():

print('\nIf this is a virtual machine, please update its CPU feature configuration or upgrade to a newer hypervisor.')
sys.exit(1)

+def configure_aio_slots():
+ with open('/proc/sys/fs/aio-max-nr') as f:
+ aio_max_nr = int(f.read())
+ required_aio_slots = cpu_count() * 11026
+ if aio_max_nr < required_aio_slots:
+ with open('/proc/sys/fs/aio-max-nr', 'w') as f:
+ f.write(str(required_aio_slots))
+
if __name__ == '__main__':
verify_cpu()

@@ -113,6 +122,8 @@ if __name__ == '__main__':

@@ -205,7 +205,6 @@ if Scylla is the main application on your server and you wish to optimize its la