[scylladb-dev] [PATCH v1] scylla_prepare: correctly handle a former 'MQ' mode
1 view
Skip to first unread message
Vlad Zolotarov
<vladz@scylladb.com>Oct 3, 2022, 8:42:42 PM10/3/22
to scylladb-dev@googlegroups.com, Vlad Zolotarov
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
---
dist/common/scripts/scylla_prepare | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
index d199903b2..48b2d7230 100755
--- a/dist/common/scripts/scylla_prepare
+++ b/dist/common/scripts/scylla_prepare
@@ -21,6 +21,20 @@ def get_cur_cpuset():
cpuset = cfg.get('CPUSET')
return re.sub(r'^--cpuset (.+)$', r'\1', cpuset).strip()
+def cpu_mask_is_zero(cpu_mask):
+ """
+ The cpu_mask is a comma-separated list of 32-bit hex values with possibly omitted zero components,
+ e.g. 0xffff,,0xffff
+ We want to estimate if the whole mask is all-zeros.
+ :param cpu_mask: hwloc-calc generated CPU mask
+ :return: True if mask is zero, False otherwise
+ """
+ for cur_cpu_mask in cpu_mask.split(','):
+ if cur_cpu_mask and int(cur_cpu_mask, 16) != 0:
+ return False
+
+ return True
+
def get_irq_cpu_mask():
"""
Return an irq_cpu_mask corresponding to a value written in cpuset.conf
@@ -60,7 +74,14 @@ def get_irq_cpu_mask():
hwloc_cmd = "/opt/scylladb/bin/hwloc-calc --pi all {}".\
format(" ".join(['~PU:{}'.format(c) for c in cur_cpuset.split(",")]))
- return out(hwloc_cmd).strip()
+ irq_cpu_mask = out(hwloc_cmd).strip()
+
+ # If the generated mask turns out to be all-zeros then it means that all present CPUs are used in cpuset.conf.
+ # In such a case irq_cpu_mask has to be all-CPUs too, a.k.a. MQ mode.
+ if cpu_mask_is_zero(irq_cpu_mask):
+ irq_cpu_mask = out("/opt/scylladb/bin/hwloc-calc all").strip()
+
+ return irq_cpu_mask
def create_perftune_conf(cfg):
"""
--
2.34.1
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
---
dist/common/scripts/scylla_prepare | 23 ++++++++++++++++++++++-
1 file changed, 22 insertions(+), 1 deletion(-)
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
index d199903b2..48b2d7230 100755
--- a/dist/common/scripts/scylla_prepare
+++ b/dist/common/scripts/scylla_prepare
@@ -21,6 +21,20 @@ def get_cur_cpuset():
cpuset = cfg.get('CPUSET')
return re.sub(r'^--cpuset (.+)$', r'\1', cpuset).strip()
+def cpu_mask_is_zero(cpu_mask):
+ """
+ The cpu_mask is a comma-separated list of 32-bit hex values with possibly omitted zero components,
+ e.g. 0xffff,,0xffff
+ We want to estimate if the whole mask is all-zeros.
+ :param cpu_mask: hwloc-calc generated CPU mask
+ :return: True if mask is zero, False otherwise
+ """
+ for cur_cpu_mask in cpu_mask.split(','):
+ if cur_cpu_mask and int(cur_cpu_mask, 16) != 0:
+ return False
+
+ return True
+
def get_irq_cpu_mask():
"""
Return an irq_cpu_mask corresponding to a value written in cpuset.conf
@@ -60,7 +74,14 @@ def get_irq_cpu_mask():
hwloc_cmd = "/opt/scylladb/bin/hwloc-calc --pi all {}".\
format(" ".join(['~PU:{}'.format(c) for c in cur_cpuset.split(",")]))
- return out(hwloc_cmd).strip()
+ irq_cpu_mask = out(hwloc_cmd).strip()
+
+ # If the generated mask turns out to be all-zeros then it means that all present CPUs are used in cpuset.conf.
+ # In such a case irq_cpu_mask has to be all-CPUs too, a.k.a. MQ mode.
+ if cpu_mask_is_zero(irq_cpu_mask):
+ irq_cpu_mask = out("/opt/scylladb/bin/hwloc-calc all").strip()
+
+ return irq_cpu_mask
def create_perftune_conf(cfg):
"""
--
2.34.1
Commit Bot
<bot@cloudius-systems.com>Oct 6, 2022, 10:43:44 AM10/6/22
to scylladb-dev@googlegroups.com, Vlad Zolotarov
From: Vlad Zolotarov <vl...@scylladb.com>
Committer: Avi Kivity <a...@scylladb.com>
Branch: next
scylla_prepare: correctly handle a former 'MQ' mode
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
Message-Id: <20221004004237....@scylladb.com>
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
Committer: Avi Kivity <a...@scylladb.com>
Branch: next
scylla_prepare: correctly handle a former 'MQ' mode
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
Commit Bot
<bot@cloudius-systems.com>Oct 6, 2022, 8:35:53 PM10/6/22
to scylladb-dev@googlegroups.com, Vlad Zolotarov
Branch: master
scylla_prepare: correctly handle a former 'MQ' mode
scylla_prepare: correctly handle a former 'MQ' mode
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Fixes #11701
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
Message-Id: <20221004004237....@scylladb.com>
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
Commit Bot
<bot@cloudius-systems.com>Jun 4, 2023, 12:26:40 PM6/4/23
to scylladb-dev@googlegroups.com, Vlad Zolotarov
Branch: next-5.1
scylla_prepare: correctly handle a former 'MQ' mode
(cherry picked from commit 8195dab92a389451e0ef24b17dcacabf0a8bb580)
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
scylla_prepare: correctly handle a former 'MQ' mode
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
\Fixes scylladb/scylladb#11701
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
Message-Id: <20221004004237....@scylladb.com>
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
(cherry picked from commit 8195dab92a389451e0ef24b17dcacabf0a8bb580)
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
Commit Bot
<bot@cloudius-systems.com>Jun 4, 2023, 3:53:54 PM6/4/23
to scylladb-dev@googlegroups.com, Vlad Zolotarov
Branch: branch-5.1
scylla_prepare: correctly handle a former 'MQ' mode
scylla_prepare: correctly handle a former 'MQ' mode
Fixes a regression introduced in 80917a1054f305d08a60107d868e86757ca3ee69:
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
\Fixes scylladb/scylladb#11701
"scylla_prepare: stop generating 'mode' value in perftune.yaml"
When cpuset.conf contains a "full" CPU set the negation of it from
the "full" CPU set is going to generate a zero mask as a irq_cpu_mask.
This is an illegal value that will eventually end up in the generated
perftune.yaml, which in line will make the scylla service fail to start
until the issue is resolved.
In such a case a irq_cpu_mask must represent a "full" CPU set mimicking
a former 'MQ' mode.
Tested:
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
- Manually on a 2 vCPU VM in an 'auto-selection' mode.
- Manually on a large VM (48 vCPUs) with an 'MQ' manually
enforced.
Message-Id: <20221004004237....@scylladb.com>
(cherry picked from commit 8195dab92a389451e0ef24b17dcacabf0a8bb580)
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
(cherry picked from commit 8195dab92a389451e0ef24b17dcacabf0a8bb580)
---
diff --git a/dist/common/scripts/scylla_prepare b/dist/common/scripts/scylla_prepare
Reply all
Reply to author
Forward
0 new messages