[PATCH v2 00/44] cpuidle,rcu: Clean up the mess

2 views
Skip to first unread message

Peter Zijlstra

unread,
Sep 19, 2022, 6:17:17 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Hi All!

At long last, a respin of the cpuidle vs rcu cleanup patches.

v1: https://lkml.kernel.org/r/202206081427...@infradead.org

These here patches clean up the mess that is cpuidle vs rcuidle.

At the end of the ride there's only on RCU_NONIDLE user left:

arch/arm64/kernel/suspend.c: RCU_NONIDLE(__cpu_suspend_exit());

and 'one' trace_*_rcuidle() user:

kernel/trace/trace_preemptirq.c: trace_irq_enable_rcuidle(CALLER_ADDR0, CALLER_ADDR1);
kernel/trace/trace_preemptirq.c: trace_irq_disable_rcuidle(CALLER_ADDR0, CALLER_ADDR1);
kernel/trace/trace_preemptirq.c: trace_irq_enable_rcuidle(CALLER_ADDR0, caller_addr);
kernel/trace/trace_preemptirq.c: trace_irq_disable_rcuidle(CALLER_ADDR0, caller_addr);
kernel/trace/trace_preemptirq.c: trace_preempt_enable_rcuidle(a0, a1);
kernel/trace/trace_preemptirq.c: trace_preempt_disable_rcuidle(a0, a1);

However this last is all in deprecated code that should be unused for GENERIC_ENTRY.

I've touched a lot of code that I can't test and I might've broken something by
accident. In particular the whole ARM cpuidle stuff was quite involved.

Please all; have a look where you haven't already.


New since v1:

- rebase on top of Frederic's rcu-context-tracking rename fest
- more omap goodness as per the last discusion (thanks Tony!)
- removed one more RCU_NONIDLE() from arm64/risc-v perf code
- ubsan/kasan fixes
- intel_idle module-param for testing
- a bunch of extra __always_inline, because compilers are silly.

---
arch/alpha/kernel/process.c | 1 -
arch/alpha/kernel/vmlinux.lds.S | 1 -
arch/arc/kernel/process.c | 3 ++
arch/arc/kernel/vmlinux.lds.S | 1 -
arch/arm/include/asm/vmlinux.lds.h | 1 -
arch/arm/kernel/process.c | 1 -
arch/arm/kernel/smp.c | 6 +--
arch/arm/mach-gemini/board-dt.c | 3 +-
arch/arm/mach-imx/cpuidle-imx6q.c | 4 +-
arch/arm/mach-imx/cpuidle-imx6sx.c | 5 ++-
arch/arm/mach-omap2/common.h | 6 ++-
arch/arm/mach-omap2/cpuidle34xx.c | 16 +++++++-
arch/arm/mach-omap2/cpuidle44xx.c | 29 +++++++-------
arch/arm/mach-omap2/omap-mpuss-lowpower.c | 12 +++++-
arch/arm/mach-omap2/pm.h | 2 +-
arch/arm/mach-omap2/pm24xx.c | 51 +-----------------------
arch/arm/mach-omap2/pm34xx.c | 14 +++++--
arch/arm/mach-omap2/pm44xx.c | 2 +-
arch/arm/mach-omap2/powerdomain.c | 10 ++---
arch/arm64/kernel/idle.c | 1 -
arch/arm64/kernel/smp.c | 4 +-
arch/arm64/kernel/vmlinux.lds.S | 1 -
arch/csky/kernel/process.c | 1 -
arch/csky/kernel/smp.c | 2 +-
arch/csky/kernel/vmlinux.lds.S | 1 -
arch/hexagon/kernel/process.c | 1 -
arch/hexagon/kernel/vmlinux.lds.S | 1 -
arch/ia64/kernel/process.c | 1 +
arch/ia64/kernel/vmlinux.lds.S | 1 -
arch/loongarch/kernel/idle.c | 1 +
arch/loongarch/kernel/vmlinux.lds.S | 1 -
arch/m68k/kernel/vmlinux-nommu.lds | 1 -
arch/m68k/kernel/vmlinux-std.lds | 1 -
arch/m68k/kernel/vmlinux-sun3.lds | 1 -
arch/microblaze/kernel/process.c | 1 -
arch/microblaze/kernel/vmlinux.lds.S | 1 -
arch/mips/kernel/idle.c | 8 ++--
arch/mips/kernel/vmlinux.lds.S | 1 -
arch/nios2/kernel/process.c | 1 -
arch/nios2/kernel/vmlinux.lds.S | 1 -
arch/openrisc/kernel/process.c | 1 +
arch/openrisc/kernel/vmlinux.lds.S | 1 -
arch/parisc/kernel/process.c | 2 -
arch/parisc/kernel/vmlinux.lds.S | 1 -
arch/powerpc/kernel/idle.c | 5 +--
arch/powerpc/kernel/vmlinux.lds.S | 1 -
arch/riscv/kernel/process.c | 1 -
arch/riscv/kernel/vmlinux-xip.lds.S | 1 -
arch/riscv/kernel/vmlinux.lds.S | 1 -
arch/s390/kernel/idle.c | 1 -
arch/s390/kernel/vmlinux.lds.S | 1 -
arch/sh/kernel/idle.c | 1 +
arch/sh/kernel/vmlinux.lds.S | 1 -
arch/sparc/kernel/leon_pmc.c | 4 ++
arch/sparc/kernel/process_32.c | 1 -
arch/sparc/kernel/process_64.c | 3 +-
arch/sparc/kernel/vmlinux.lds.S | 1 -
arch/um/kernel/dyn.lds.S | 1 -
arch/um/kernel/process.c | 1 -
arch/um/kernel/uml.lds.S | 1 -
arch/x86/boot/compressed/vmlinux.lds.S | 1 +
arch/x86/coco/tdx/tdcall.S | 15 +------
arch/x86/coco/tdx/tdx.c | 25 ++++--------
arch/x86/events/amd/brs.c | 13 +++----
arch/x86/include/asm/fpu/xcr.h | 4 +-
arch/x86/include/asm/irqflags.h | 11 ++----
arch/x86/include/asm/mwait.h | 14 +++----
arch/x86/include/asm/nospec-branch.h | 2 +-
arch/x86/include/asm/paravirt.h | 6 ++-
arch/x86/include/asm/perf_event.h | 2 +-
arch/x86/include/asm/shared/io.h | 4 +-
arch/x86/include/asm/shared/tdx.h | 1 -
arch/x86/include/asm/special_insns.h | 8 ++--
arch/x86/include/asm/xen/hypercall.h | 2 +-
arch/x86/kernel/cpu/bugs.c | 2 +-
arch/x86/kernel/fpu/core.c | 4 +-
arch/x86/kernel/paravirt.c | 14 ++++++-
arch/x86/kernel/process.c | 65 +++++++++++++++----------------
arch/x86/kernel/vmlinux.lds.S | 1 -
arch/x86/lib/memcpy_64.S | 5 +--
arch/x86/lib/memmove_64.S | 4 +-
arch/x86/lib/memset_64.S | 4 +-
arch/x86/xen/enlighten_pv.c | 2 +-
arch/x86/xen/irq.c | 2 +-
arch/xtensa/kernel/process.c | 1 +
arch/xtensa/kernel/vmlinux.lds.S | 1 -
drivers/acpi/processor_idle.c | 36 ++++++++++-------
drivers/base/power/runtime.c | 24 ++++++------
drivers/clk/clk.c | 8 ++--
drivers/cpuidle/cpuidle-arm.c | 1 +
drivers/cpuidle/cpuidle-big_little.c | 8 +++-
drivers/cpuidle/cpuidle-mvebu-v7.c | 7 ++++
drivers/cpuidle/cpuidle-psci.c | 10 +++--
drivers/cpuidle/cpuidle-qcom-spm.c | 1 +
drivers/cpuidle/cpuidle-riscv-sbi.c | 10 +++--
drivers/cpuidle/cpuidle-tegra.c | 21 +++++++---
drivers/cpuidle/cpuidle.c | 21 +++++-----
drivers/cpuidle/dt_idle_states.c | 2 +-
drivers/cpuidle/poll_state.c | 10 ++++-
drivers/idle/intel_idle.c | 19 +++++----
drivers/perf/arm_pmu.c | 11 +-----
drivers/perf/riscv_pmu_sbi.c | 8 +---
include/asm-generic/vmlinux.lds.h | 9 ++---
include/linux/compiler_types.h | 8 +++-
include/linux/cpu.h | 3 --
include/linux/cpuidle.h | 34 ++++++++++++++++
include/linux/cpumask.h | 4 +-
include/linux/percpu-defs.h | 2 +-
include/linux/sched/idle.h | 40 ++++++++++++++-----
include/linux/thread_info.h | 18 ++++++++-
include/linux/tracepoint.h | 13 ++++++-
kernel/cpu_pm.c | 9 -----
kernel/printk/printk.c | 2 +-
kernel/sched/idle.c | 47 +++++++---------------
kernel/time/tick-broadcast-hrtimer.c | 29 ++++++--------
kernel/time/tick-broadcast.c | 6 ++-
kernel/trace/trace.c | 3 ++
lib/ubsan.c | 5 ++-
mm/kasan/kasan.h | 4 ++
mm/kasan/shadow.c | 38 ++++++++++++++++++
tools/objtool/check.c | 17 ++++++++
121 files changed, 511 insertions(+), 420 deletions(-)

Peter Zijlstra

unread,
Sep 19, 2022, 6:17:18 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Make cpuidle_enter_state() consistent with the s2idle variant and
verify ->enter() always returns with interrupts disabled.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/cpuidle/cpuidle.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)

--- a/drivers/cpuidle/cpuidle.c
+++ b/drivers/cpuidle/cpuidle.c
@@ -236,7 +236,11 @@ int cpuidle_enter_state(struct cpuidle_d
stop_critical_timings();
if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
ct_idle_enter();
+
entered_state = target_state->enter(dev, drv, index);
+ if (WARN_ONCE(!irqs_disabled(), "%ps leaked IRQ state", target_state->enter))
+ raw_local_irq_disable();
+
if (!(target_state->flags & CPUIDLE_FLAG_RCU_IDLE))
ct_idle_exit();
start_critical_timings();
@@ -248,12 +252,8 @@ int cpuidle_enter_state(struct cpuidle_d
/* The cpu is no longer idle or about to enter idle. */
sched_idle_set_state(NULL);

- if (broadcast) {
- if (WARN_ON_ONCE(!irqs_disabled()))
- local_irq_disable();
-
+ if (broadcast)
tick_broadcast_exit();
- }

if (!cpuidle_state_is_coupled(drv, index))
local_irq_enable();


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:18 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/arm/mach-imx/cpuidle-imx6sx.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)

--- a/arch/arm/mach-imx/cpuidle-imx6sx.c
+++ b/arch/arm/mach-imx/cpuidle-imx6sx.c
@@ -47,7 +47,9 @@ static int imx6sx_enter_wait(struct cpui
cpu_pm_enter();
cpu_cluster_pm_enter();

+ ct_idle_enter();
cpu_suspend(0, imx6sx_idle_finish);
+ ct_idle_exit();

cpu_cluster_pm_exit();
cpu_pm_exit();
@@ -87,7 +89,8 @@ static struct cpuidle_driver imx6sx_cpui
*/
.exit_latency = 300,
.target_residency = 500,
- .flags = CPUIDLE_FLAG_TIMER_STOP,
+ .flags = CPUIDLE_FLAG_TIMER_STOP |
+ CPUIDLE_FLAG_RCU_IDLE,
.enter = imx6sx_enter_wait,
.name = "LOW-POWER-IDLE",
.desc = "ARM power off",


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:19 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/cpuidle/cpuidle-tegra.c | 21 ++++++++++++++++-----
1 file changed, 16 insertions(+), 5 deletions(-)

--- a/drivers/cpuidle/cpuidle-tegra.c
+++ b/drivers/cpuidle/cpuidle-tegra.c
@@ -180,9 +180,11 @@ static int tegra_cpuidle_state_enter(str
}

local_fiq_disable();
- RCU_NONIDLE(tegra_pm_set_cpu_in_lp2());
+ tegra_pm_set_cpu_in_lp2();
cpu_pm_enter();

+ ct_idle_enter();
+
switch (index) {
case TEGRA_C7:
err = tegra_cpuidle_c7_enter();
@@ -197,8 +199,10 @@ static int tegra_cpuidle_state_enter(str
break;
}

+ ct_idle_exit();
+
cpu_pm_exit();
- RCU_NONIDLE(tegra_pm_clear_cpu_in_lp2());
+ tegra_pm_clear_cpu_in_lp2();
local_fiq_enable();

return err ?: index;
@@ -226,6 +230,7 @@ static int tegra_cpuidle_enter(struct cp
struct cpuidle_driver *drv,
int index)
{
+ bool do_rcu = drv->states[index].flags & CPUIDLE_FLAG_RCU_IDLE;
unsigned int cpu = cpu_logical_map(dev->cpu);
int ret;

@@ -233,9 +238,13 @@ static int tegra_cpuidle_enter(struct cp
if (dev->states_usage[index].disable)
return -1;

- if (index == TEGRA_C1)
+ if (index == TEGRA_C1) {
+ if (do_rcu)
+ ct_idle_enter();
ret = arm_cpuidle_simple_enter(dev, drv, index);
- else
+ if (do_rcu)
+ ct_idle_exit();
+ } else
ret = tegra_cpuidle_state_enter(dev, index, cpu);

if (ret < 0) {
@@ -285,7 +294,8 @@ static struct cpuidle_driver tegra_idle_
.exit_latency = 2000,
.target_residency = 2200,
.power_usage = 100,
- .flags = CPUIDLE_FLAG_TIMER_STOP,
+ .flags = CPUIDLE_FLAG_TIMER_STOP |
+ CPUIDLE_FLAG_RCU_IDLE,
.name = "C7",
.desc = "CPU core powered off",
},
@@ -295,6 +305,7 @@ static struct cpuidle_driver tegra_idle_
.target_residency = 10000,
.power_usage = 0,
.flags = CPUIDLE_FLAG_TIMER_STOP |
+ CPUIDLE_FLAG_RCU_IDLE |
CPUIDLE_FLAG_COUPLED,
.name = "CC6",
.desc = "CPU cluster powered off",


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:19 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com, Frederic Weisbecker, Rafael J. Wysocki
Typical boot time setup; no need to suffer an indirect call for that.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Frederic Weisbecker <fred...@kernel.org>
Reviewed-by: Rafael J. Wysocki <rafael.j...@intel.com>
---
arch/x86/kernel/process.c | 50 +++++++++++++++++++++++++---------------------
1 file changed, 28 insertions(+), 22 deletions(-)

--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -24,6 +24,7 @@
#include <linux/cpuidle.h>
#include <linux/acpi.h>
#include <linux/elf-randomize.h>
+#include <linux/static_call.h>
#include <trace/events/power.h>
#include <linux/hw_breakpoint.h>
#include <asm/cpu.h>
@@ -692,7 +693,23 @@ void __switch_to_xtra(struct task_struct
unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
EXPORT_SYMBOL(boot_option_idle_override);

-static void (*x86_idle)(void);
+/*
+ * We use this if we don't have any better idle routine..
+ */
+void __cpuidle default_idle(void)
+{
+ raw_safe_halt();
+}
+#if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE)
+EXPORT_SYMBOL(default_idle);
+#endif
+
+DEFINE_STATIC_CALL_NULL(x86_idle, default_idle);
+
+static bool x86_idle_set(void)
+{
+ return !!static_call_query(x86_idle);
+}

#ifndef CONFIG_SMP
static inline void play_dead(void)
@@ -715,28 +732,17 @@ void arch_cpu_idle_dead(void)
/*
* Called from the generic idle code.
*/
-void arch_cpu_idle(void)
-{
- x86_idle();
-}
-
-/*
- * We use this if we don't have any better idle routine..
- */
-void __cpuidle default_idle(void)
+void __cpuidle arch_cpu_idle(void)
{
- raw_safe_halt();
+ static_call(x86_idle)();
}
-#if defined(CONFIG_APM_MODULE) || defined(CONFIG_HALTPOLL_CPUIDLE_MODULE)
-EXPORT_SYMBOL(default_idle);
-#endif

#ifdef CONFIG_XEN
bool xen_set_default_idle(void)
{
- bool ret = !!x86_idle;
+ bool ret = x86_idle_set();

- x86_idle = default_idle;
+ static_call_update(x86_idle, default_idle);

return ret;
}
@@ -859,20 +865,20 @@ void select_idle_routine(const struct cp
if (boot_option_idle_override == IDLE_POLL && smp_num_siblings > 1)
pr_warn_once("WARNING: polling idle and HT enabled, performance may degrade\n");
#endif
- if (x86_idle || boot_option_idle_override == IDLE_POLL)
+ if (x86_idle_set() || boot_option_idle_override == IDLE_POLL)
return;

if (boot_cpu_has_bug(X86_BUG_AMD_E400)) {
pr_info("using AMD E400 aware idle routine\n");
- x86_idle = amd_e400_idle;
+ static_call_update(x86_idle, amd_e400_idle);
} else if (prefer_mwait_c1_over_halt(c)) {
pr_info("using mwait in idle threads\n");
- x86_idle = mwait_idle;
+ static_call_update(x86_idle, mwait_idle);
} else if (cpu_feature_enabled(X86_FEATURE_TDX_GUEST)) {
pr_info("using TDX aware idle routine\n");
- x86_idle = tdx_safe_halt;
+ static_call_update(x86_idle, tdx_safe_halt);
} else
- x86_idle = default_idle;
+ static_call_update(x86_idle, default_idle);
}

void amd_e400_c1e_apic_setup(void)
@@ -925,7 +931,7 @@ static int __init idle_setup(char *str)
* To continue to load the CPU idle driver, don't touch
* the boot_option_idle_override.
*/
- x86_idle = default_idle;
+ static_call_update(x86_idle, default_idle);
boot_option_idle_override = IDLE_HALT;
} else if (!strcmp(str, "nomwait")) {
/*


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:22 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com, Rafael J. Wysocki
cpuidle_state::enter() methods should be IRQ invariant

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Rafael J. Wysocki <rafael.j...@intel.com>
---
drivers/cpuidle/poll_state.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)

--- a/drivers/cpuidle/poll_state.c
+++ b/drivers/cpuidle/poll_state.c
@@ -17,7 +17,7 @@ static int __cpuidle poll_idle(struct cp

dev->poll_time_limit = false;

- local_irq_enable();
+ raw_local_irq_enable();
if (!current_set_polling_and_test()) {
unsigned int loop_count = 0;
u64 limit;
@@ -36,6 +36,8 @@ static int __cpuidle poll_idle(struct cp
}
}
}
+ raw_local_irq_disable();
+
current_clr_polling();

return index;


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:22 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then teporarily enable it
again before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Tony Lindgren <to...@atomide.com>
Tested-by: Tony Lindgren <to...@atomide.com>
---
arch/arm/mach-omap2/cpuidle34xx.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)

--- a/arch/arm/mach-omap2/cpuidle34xx.c
+++ b/arch/arm/mach-omap2/cpuidle34xx.c
@@ -133,7 +133,9 @@ static int omap3_enter_idle(struct cpuid
}

/* Execute ARM wfi */
+ ct_idle_enter();
omap_sram_idle();
+ ct_idle_exit();

/*
* Call idle CPU PM enter notifier chain to restore
@@ -265,6 +267,7 @@ static struct cpuidle_driver omap3_idle_
.owner = THIS_MODULE,
.states = {
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 2 + 2,
.target_residency = 5,
@@ -272,6 +275,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU ON + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 10 + 10,
.target_residency = 30,
@@ -279,6 +283,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU ON + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 50 + 50,
.target_residency = 300,
@@ -286,6 +291,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU RET + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 1500 + 1800,
.target_residency = 4000,
@@ -293,6 +299,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU OFF + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 2500 + 7500,
.target_residency = 12000,
@@ -300,6 +307,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU RET + CORE RET",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 3000 + 8500,
.target_residency = 15000,
@@ -307,6 +315,7 @@ static struct cpuidle_driver omap3_idle_
.desc = "MPU OFF + CORE RET",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 10000 + 30000,
.target_residency = 30000,
@@ -328,6 +337,7 @@ static struct cpuidle_driver omap3430_id
.owner = THIS_MODULE,
.states = {
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 110 + 162,
.target_residency = 5,
@@ -335,6 +345,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU ON + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 106 + 180,
.target_residency = 309,
@@ -342,6 +353,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU ON + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 107 + 410,
.target_residency = 46057,
@@ -349,6 +361,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU RET + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 121 + 3374,
.target_residency = 46057,
@@ -356,6 +369,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU OFF + CORE ON",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 855 + 1146,
.target_residency = 46057,
@@ -363,6 +377,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU RET + CORE RET",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 7580 + 4134,
.target_residency = 484329,
@@ -370,6 +385,7 @@ static struct cpuidle_driver omap3430_id
.desc = "MPU OFF + CORE RET",
},
{
+ .flags = CPUIDLE_FLAG_RCU_IDLE,
.enter = omap3_enter_idle_bm,
.exit_latency = 7505 + 15274,
.target_residency = 484329,


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:27 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
All the idle routines are called with RCU disabled, as such there must
not be any tracing inside.

While there; clean-up the io-port idle thing.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/acpi/processor_idle.c | 24 +++++++++++++-----------
1 file changed, 13 insertions(+), 11 deletions(-)

--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -108,8 +108,8 @@ static const struct dmi_system_id proces
static void __cpuidle acpi_safe_halt(void)
{
if (!tif_need_resched()) {
- safe_halt();
- local_irq_disable();
+ raw_safe_halt();
+ raw_local_irq_disable();
}
}

@@ -524,16 +524,21 @@ static int acpi_idle_bm_check(void)
return bm_status;
}

-static void wait_for_freeze(void)
+static __cpuidle void io_idle(unsigned long addr)
{
+ /* IO port based C-state */
+ inb(addr);
+
#ifdef CONFIG_X86
/* No delay is needed if we are in guest */
if (boot_cpu_has(X86_FEATURE_HYPERVISOR))
return;
#endif
- /* Dummy wait op - must do something useless after P_LVL2 read
- because chipsets cannot guarantee that STPCLK# signal
- gets asserted in time to freeze execution properly. */
+ /*
+ * Dummy wait op - must do something useless after P_LVL2 read
+ * because chipsets cannot guarantee that STPCLK# signal
+ * gets asserted in time to freeze execution properly.
+ */
inl(acpi_gbl_FADT.xpm_timer_block.address);
}

@@ -553,9 +558,7 @@ static void __cpuidle acpi_idle_do_entry
} else if (cx->entry_method == ACPI_CSTATE_HALT) {
acpi_safe_halt();
} else {
- /* IO port based C-state */
- inb(cx->address);
- wait_for_freeze();
+ io_idle(cx->address);
}

perf_lopwr_cb(false);
@@ -577,8 +580,7 @@ static int acpi_idle_play_dead(struct cp
if (cx->entry_method == ACPI_CSTATE_HALT)
safe_halt();
else if (cx->entry_method == ACPI_CSTATE_SYSTEMIO) {
- inb(cx->address);
- wait_for_freeze();
+ io_idle(cx->address);
} else
return -ENODEV;



Peter Zijlstra

unread,
Sep 19, 2022, 6:17:28 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/cpuidle/cpuidle-psci.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)

--- a/drivers/cpuidle/cpuidle-psci.c
+++ b/drivers/cpuidle/cpuidle-psci.c
@@ -69,12 +69,12 @@ static int __psci_enter_domain_idle_stat
return -1;

/* Do runtime PM to manage a hierarchical CPU toplogy. */
- ct_irq_enter_irqson();
if (s2idle)
dev_pm_genpd_suspend(pd_dev);
else
pm_runtime_put_sync_suspend(pd_dev);
- ct_irq_exit_irqson();
+
+ ct_idle_enter();

state = psci_get_domain_state();
if (!state)
@@ -82,12 +82,12 @@ static int __psci_enter_domain_idle_stat

ret = psci_cpu_suspend_enter(state) ? -1 : idx;

- ct_irq_enter_irqson();
+ ct_idle_exit();
+
if (s2idle)
dev_pm_genpd_resume(pd_dev);
else
pm_runtime_get_sync(pd_dev);
- ct_irq_exit_irqson();

cpu_pm_exit();

@@ -240,6 +240,7 @@ static int psci_dt_cpu_init_topology(str
* of a shared state for the domain, assumes the domain states are all
* deeper states.
*/
+ drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE;
drv->states[state_count - 1].enter = psci_enter_domain_idle_state;
drv->states[state_count - 1].enter_s2idle = psci_enter_s2idle_domain_idle_state;
psci_cpuidle_use_cpuhp = true;


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:35 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: intel_idle_irq+0x10c: call to trace_hardirqs_off() leaves .noinstr.text section

As per commit 32d4fd5751ea ("cpuidle,intel_idle: Fix
CPUIDLE_FLAG_IRQ_ENABLE"):

"must not have tracing in idle functions"

Clearly people can't read and tinker along until splat dissapears.
This straight up reverts commit d295ad34f236 ("intel_idle: Fix false
positive RCU splats due to incorrect hardirqs state").

It doesn't re-introduce the problem because preceding patches fixed it
properly.

Fixes: d295ad34f236 ("intel_idle: Fix false positive RCU splats due to incorrect hardirqs state")
Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/idle/intel_idle.c | 8 +-------
1 file changed, 1 insertion(+), 7 deletions(-)

--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -168,13 +168,7 @@ static __cpuidle int intel_idle_irq(stru

raw_local_irq_enable();
ret = __intel_idle(dev, drv, index);
-
- /*
- * The lockdep hardirqs state may be changed to 'on' with timer
- * tick interrupt followed by __do_softirq(). Use local_irq_disable()
- * to keep the hardirqs state correct.
- */
- local_irq_disable();
+ raw_local_irq_disable();

return ret;
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:35 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: intel_idle_ibrs+0x17: call to spec_ctrl_current() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_ibrs+0x27: call to wrmsrl.constprop.0() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/kernel/cpu/bugs.c | 2 +-
drivers/idle/intel_idle.c | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)

--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -79,7 +79,7 @@ void write_spec_ctrl_current(u64 val, bo
wrmsrl(MSR_IA32_SPEC_CTRL, val);
}

-u64 spec_ctrl_current(void)
+noinstr u64 spec_ctrl_current(void)
{
return this_cpu_read(x86_spec_ctrl_current);
}
--- a/drivers/idle/intel_idle.c
+++ b/drivers/idle/intel_idle.c
@@ -181,12 +181,12 @@ static __cpuidle int intel_idle_ibrs(str
int ret;

if (smt_active)
- wrmsrl(MSR_IA32_SPEC_CTRL, 0);
+ native_wrmsrl(MSR_IA32_SPEC_CTRL, 0);

ret = __intel_idle(dev, drv, index);

if (smt_active)
- wrmsrl(MSR_IA32_SPEC_CTRL, spec_ctrl);
+ native_wrmsrl(MSR_IA32_SPEC_CTRL, spec_ctrl);

return ret;
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:35 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com, Rafael J. Wysocki
The __cpuidle functions will become a noinstr class, as such they need
explicit annotations.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Rafael J. Wysocki <rafael.j...@intel.com>
---
drivers/cpuidle/poll_state.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

--- a/drivers/cpuidle/poll_state.c
+++ b/drivers/cpuidle/poll_state.c
@@ -13,7 +13,10 @@
static int __cpuidle poll_idle(struct cpuidle_device *dev,
struct cpuidle_driver *drv, int index)
{
- u64 time_start = local_clock();
+ u64 time_start;
+
+ instrumentation_begin();
+ time_start = local_clock();

dev->poll_time_limit = false;

@@ -39,6 +42,7 @@ static int __cpuidle poll_idle(struct cp
raw_local_irq_disable();

current_clr_polling();
+ instrumentation_end();

return index;
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:37 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: __halt+0x2c: call to hcall_func.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: __halt+0x3f: call to __tdx_hypercall() leaves .noinstr.text section
vmlinux.o: warning: objtool: __tdx_hypercall+0x66: call to __tdx_hypercall_failed() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/boot/compressed/vmlinux.lds.S | 1 +
arch/x86/coco/tdx/tdcall.S | 2 ++
arch/x86/coco/tdx/tdx.c | 5 +++--
3 files changed, 6 insertions(+), 2 deletions(-)

--- a/arch/x86/boot/compressed/vmlinux.lds.S
+++ b/arch/x86/boot/compressed/vmlinux.lds.S
@@ -34,6 +34,7 @@ SECTIONS
_text = .; /* Text */
*(.text)
*(.text.*)
+ *(.noinstr.text)
_etext = . ;
}
.rodata : {
--- a/arch/x86/coco/tdx/tdcall.S
+++ b/arch/x86/coco/tdx/tdcall.S
@@ -31,6 +31,8 @@
TDX_R12 | TDX_R13 | \
TDX_R14 | TDX_R15 )

+.section .noinstr.text, "ax"
+
/*
* __tdx_module_call() - Used by TDX guests to request services from
* the TDX module (does not include VMM services) using TDCALL instruction.
--- a/arch/x86/coco/tdx/tdx.c
+++ b/arch/x86/coco/tdx/tdx.c
@@ -53,8 +53,9 @@ static inline u64 _tdx_hypercall(u64 fn,
}

/* Called from __tdx_hypercall() for unrecoverable failure */
-void __tdx_hypercall_failed(void)
+noinstr void __tdx_hypercall_failed(void)
{
+ instrumentation_begin();
panic("TDVMCALL failed. TDX module bug?");
}

@@ -64,7 +65,7 @@ void __tdx_hypercall_failed(void)
* Reusing the KVM EXIT_REASON macros makes it easier to connect the host and
* guest sides of these calls.
*/
-static u64 hcall_func(u64 exit_reason)
+static __always_inline u64 hcall_func(u64 exit_reason)
{
return exit_reason;
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:37 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: mwait_idle+0x5: call to current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xc5: call to current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0xbc: call to current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0xea: call to current_set_polling_and_test() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_s2idle+0xb4: call to current_set_polling_and_test() leaves .noinstr.text section

vmlinux.o: warning: objtool: intel_idle+0xa6: call to current_clr_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0xbf: call to current_clr_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_s2idle+0xa1: call to current_clr_polling() leaves .noinstr.text section

vmlinux.o: warning: objtool: mwait_idle+0xe: call to __current_set_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0xc5: call to __current_set_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0xbc: call to __current_set_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0xea: call to __current_set_polling() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_s2idle+0xb4: call to __current_set_polling() leaves .noinstr.text section

vmlinux.o: warning: objtool: cpu_idle_poll.isra.0+0x73: call to test_ti_thread_flag() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_s2idle+0x73: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0x91: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0x78: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_safe_halt+0xf: call to test_ti_thread_flag.constprop.0() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
include/linux/sched/idle.h | 40 ++++++++++++++++++++++++++++++----------
include/linux/thread_info.h | 18 +++++++++++++++++-
2 files changed, 47 insertions(+), 11 deletions(-)

--- a/include/linux/sched/idle.h
+++ b/include/linux/sched/idle.h
@@ -23,12 +23,37 @@ static inline void wake_up_if_idle(int c
*/
#ifdef TIF_POLLING_NRFLAG

-static inline void __current_set_polling(void)
+#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_ATOMIC_H
+
+static __always_inline void __current_set_polling(void)
{
- set_thread_flag(TIF_POLLING_NRFLAG);
+ arch_set_bit(TIF_POLLING_NRFLAG,
+ (unsigned long *)(&current_thread_info()->flags));
}

-static inline bool __must_check current_set_polling_and_test(void)
+static __always_inline void __current_clr_polling(void)
+{
+ arch_clear_bit(TIF_POLLING_NRFLAG,
+ (unsigned long *)(&current_thread_info()->flags));
+}
+
+#else
+
+static __always_inline void __current_set_polling(void)
+{
+ set_bit(TIF_POLLING_NRFLAG,
+ (unsigned long *)(&current_thread_info()->flags));
+}
+
+static __always_inline void __current_clr_polling(void)
+{
+ clear_bit(TIF_POLLING_NRFLAG,
+ (unsigned long *)(&current_thread_info()->flags));
+}
+
+#endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_ATOMIC_H */
+
+static __always_inline bool __must_check current_set_polling_and_test(void)
{
__current_set_polling();

@@ -41,12 +66,7 @@ static inline bool __must_check current_
return unlikely(tif_need_resched());
}

-static inline void __current_clr_polling(void)
-{
- clear_thread_flag(TIF_POLLING_NRFLAG);
-}
-
-static inline bool __must_check current_clr_polling_and_test(void)
+static __always_inline bool __must_check current_clr_polling_and_test(void)
{
__current_clr_polling();

@@ -73,7 +93,7 @@ static inline bool __must_check current_
}
#endif

-static inline void current_clr_polling(void)
+static __always_inline void current_clr_polling(void)
{
__current_clr_polling();

--- a/include/linux/thread_info.h
+++ b/include/linux/thread_info.h
@@ -177,7 +177,23 @@ static __always_inline unsigned long rea
clear_ti_thread_flag(task_thread_info(t), TIF_##fl)
#endif /* !CONFIG_GENERIC_ENTRY */

-#define tif_need_resched() test_thread_flag(TIF_NEED_RESCHED)
+#ifdef _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H
+
+static __always_inline bool tif_need_resched(void)
+{
+ return arch_test_bit(TIF_NEED_RESCHED,
+ (unsigned long *)(&current_thread_info()->flags));
+}
+
+#else
+
+static __always_inline bool tif_need_resched(void)
+{
+ return test_bit(TIF_NEED_RESCHED,
+ (unsigned long *)(&current_thread_info()->flags));
+}
+
+#endif /* _ASM_GENERIC_BITOPS_INSTRUMENTED_NON_ATOMIC_H */

#ifndef CONFIG_HAVE_ARCH_WITHIN_STACK_FRAMES
static inline int arch_within_stack_frames(const void * const stack,


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:38 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
arch_cpu_idle() is a very simple idle interface and exposes only a
single idle state and is expected to not require RCU and not do any
tracing/instrumentation.

As such, omap_sram_idle() is not a valid implementation. Replace it
with the simple (shallow) omap3_do_wfi() call. Leaving the more
complicated idle states for the cpuidle driver.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Acked-by: Tony Lindgren <to...@atomide.com>
---
arch/arm/mach-omap2/pm34xx.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

--- a/arch/arm/mach-omap2/pm34xx.c
+++ b/arch/arm/mach-omap2/pm34xx.c
@@ -294,7 +294,7 @@ static void omap3_pm_idle(void)
if (omap_irq_pending())
return;

- omap_sram_idle();
+ omap3_do_wfi();
}

#ifdef CONFIG_SUSPEND


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:39 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
From: Tony Lindgren <to...@atomide.com>

OMAP4 uses full SoC suspend modes as idle states, as such it needs the
whole power-domain and clock-domain code from the idle path.

All that code is not suitable to run with RCU disabled, as such push
RCU-idle deeper still.

Signed-off-by: Tony Lindgren <to...@atomide.com>
Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Link: https://lkml.kernel.org/r/Yqcv6crS...@atomide.com
---
arch/arm/mach-omap2/common.h | 6 ++++--
arch/arm/mach-omap2/cpuidle44xx.c | 8 ++------
arch/arm/mach-omap2/omap-mpuss-lowpower.c | 12 +++++++++++-
arch/arm/mach-omap2/pm44xx.c | 2 +-
4 files changed, 18 insertions(+), 10 deletions(-)

--- a/arch/arm/mach-omap2/common.h
+++ b/arch/arm/mach-omap2/common.h
@@ -284,11 +284,13 @@ extern u32 omap4_get_cpu1_ns_pa_addr(voi

#if defined(CONFIG_SMP) && defined(CONFIG_PM)
extern int omap4_mpuss_init(void);
-extern int omap4_enter_lowpower(unsigned int cpu, unsigned int power_state);
+extern int omap4_enter_lowpower(unsigned int cpu, unsigned int power_state,
+ bool rcuidle);
extern int omap4_hotplug_cpu(unsigned int cpu, unsigned int power_state);
#else
static inline int omap4_enter_lowpower(unsigned int cpu,
- unsigned int power_state)
+ unsigned int power_state,
+ bool rcuidle)
{
cpu_do_idle();
return 0;
--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -105,9 +105,7 @@ static int omap_enter_idle_smp(struct cp
}
raw_spin_unlock_irqrestore(&mpu_lock, flag);

- ct_cpuidle_enter();
- omap4_enter_lowpower(dev->cpu, cx->cpu_state);
- ct_cpuidle_exit();
+ omap4_enter_lowpower(dev->cpu, cx->cpu_state, true);

raw_spin_lock_irqsave(&mpu_lock, flag);
if (cx->mpu_state_vote == num_online_cpus())
@@ -186,10 +184,8 @@ static int omap_enter_idle_coupled(struc
}
}

- ct_cpuidle_enter();
- omap4_enter_lowpower(dev->cpu, cx->cpu_state);
+ omap4_enter_lowpower(dev->cpu, cx->cpu_state, true);
cpu_done[dev->cpu] = true;
- ct_cpuidle_exit();

/* Wakeup CPU1 only if it is not offlined */
if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
--- a/arch/arm/mach-omap2/omap-mpuss-lowpower.c
+++ b/arch/arm/mach-omap2/omap-mpuss-lowpower.c
@@ -33,6 +33,7 @@
* and first to wake-up when MPUSS low power states are excercised
*/

+#include <linux/cpuidle.h>
#include <linux/kernel.h>
#include <linux/io.h>
#include <linux/errno.h>
@@ -214,6 +215,7 @@ static void __init save_l2x0_context(voi
* of OMAP4 MPUSS subsystem
* @cpu : CPU ID
* @power_state: Low power state.
+ * @rcuidle: RCU needs to be idled
*
* MPUSS states for the context save:
* save_state =
@@ -222,7 +224,8 @@ static void __init save_l2x0_context(voi
* 2 - CPUx L1 and logic lost + GIC lost: MPUSS OSWR
* 3 - CPUx L1 and logic lost + GIC + L2 lost: DEVICE OFF
*/
-int omap4_enter_lowpower(unsigned int cpu, unsigned int power_state)
+int omap4_enter_lowpower(unsigned int cpu, unsigned int power_state,
+ bool rcuidle)
{
struct omap4_cpu_pm_info *pm_info = &per_cpu(omap4_pm_info, cpu);
unsigned int save_state = 0, cpu_logic_state = PWRDM_POWER_RET;
@@ -268,6 +271,10 @@ int omap4_enter_lowpower(unsigned int cp
cpu_clear_prev_logic_pwrst(cpu);
pwrdm_set_next_pwrst(pm_info->pwrdm, power_state);
pwrdm_set_logic_retst(pm_info->pwrdm, cpu_logic_state);
+
+ if (rcuidle)
+ ct_cpuidle_enter();
+
set_cpu_wakeup_addr(cpu, __pa_symbol(omap_pm_ops.resume));
omap_pm_ops.scu_prepare(cpu, power_state);
l2x0_pwrst_prepare(cpu, save_state);
@@ -283,6 +290,9 @@ int omap4_enter_lowpower(unsigned int cp
if (IS_PM44XX_ERRATUM(PM_OMAP4_ROM_SMP_BOOT_ERRATUM_GICD) && cpu)
gic_dist_enable();

+ if (rcuidle)
+ ct_cpuidle_exit();
+
/*
* Restore the CPUx power state to ON otherwise CPUx
* power domain can transitions to programmed low power
--- a/arch/arm/mach-omap2/pm44xx.c
+++ b/arch/arm/mach-omap2/pm44xx.c
@@ -76,7 +76,7 @@ static int omap4_pm_suspend(void)
* domain CSWR is not supported by hardware.
* More details can be found in OMAP4430 TRM section 4.3.4.2.
*/
- omap4_enter_lowpower(cpu_id, cpu_suspend_state);
+ omap4_enter_lowpower(cpu_id, cpu_suspend_state, false);

/* Restore next powerdomain state */
list_for_each_entry(pwrst, &pwrst_list, node) {


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:41 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
OMAP was the one and only user.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/arm/mach-omap2/powerdomain.c | 10 +++++-----
drivers/base/power/runtime.c | 24 ++++++++++++------------
2 files changed, 17 insertions(+), 17 deletions(-)

--- a/arch/arm/mach-omap2/powerdomain.c
+++ b/arch/arm/mach-omap2/powerdomain.c
@@ -187,9 +187,9 @@ static int _pwrdm_state_switch(struct po
trace_state = (PWRDM_TRACE_STATES_FLAG |
((next & OMAP_POWERSTATE_MASK) << 8) |
((prev & OMAP_POWERSTATE_MASK) << 0));
- trace_power_domain_target_rcuidle(pwrdm->name,
- trace_state,
- raw_smp_processor_id());
+ trace_power_domain_target(pwrdm->name,
+ trace_state,
+ raw_smp_processor_id());
}
break;
default:
@@ -541,8 +541,8 @@ int pwrdm_set_next_pwrst(struct powerdom

if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
/* Trace the pwrdm desired target state */
- trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
- raw_smp_processor_id());
+ trace_power_domain_target(pwrdm->name, pwrst,
+ raw_smp_processor_id());
/* Program the pwrdm desired target state */
ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
}
--- a/drivers/base/power/runtime.c
+++ b/drivers/base/power/runtime.c
@@ -442,7 +442,7 @@ static int rpm_idle(struct device *dev,
int (*callback)(struct device *);
int retval;

- trace_rpm_idle_rcuidle(dev, rpmflags);
+ trace_rpm_idle(dev, rpmflags);
retval = rpm_check_suspend_allowed(dev);
if (retval < 0)
; /* Conditions are wrong. */
@@ -481,7 +481,7 @@ static int rpm_idle(struct device *dev,
dev->power.request_pending = true;
queue_work(pm_wq, &dev->power.work);
}
- trace_rpm_return_int_rcuidle(dev, _THIS_IP_, 0);
+ trace_rpm_return_int(dev, _THIS_IP_, 0);
return 0;
}

@@ -493,7 +493,7 @@ static int rpm_idle(struct device *dev,
wake_up_all(&dev->power.wait_queue);

out:
- trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+ trace_rpm_return_int(dev, _THIS_IP_, retval);
return retval ? retval : rpm_suspend(dev, rpmflags | RPM_AUTO);
}

@@ -557,7 +557,7 @@ static int rpm_suspend(struct device *de
struct device *parent = NULL;
int retval;

- trace_rpm_suspend_rcuidle(dev, rpmflags);
+ trace_rpm_suspend(dev, rpmflags);

repeat:
retval = rpm_check_suspend_allowed(dev);
@@ -708,7 +708,7 @@ static int rpm_suspend(struct device *de
}

out:
- trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+ trace_rpm_return_int(dev, _THIS_IP_, retval);

return retval;

@@ -760,7 +760,7 @@ static int rpm_resume(struct device *dev
struct device *parent = NULL;
int retval = 0;

- trace_rpm_resume_rcuidle(dev, rpmflags);
+ trace_rpm_resume(dev, rpmflags);

repeat:
if (dev->power.runtime_error) {
@@ -925,7 +925,7 @@ static int rpm_resume(struct device *dev
spin_lock_irq(&dev->power.lock);
}

- trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+ trace_rpm_return_int(dev, _THIS_IP_, retval);

return retval;
}
@@ -1081,7 +1081,7 @@ int __pm_runtime_idle(struct device *dev
if (retval < 0) {
return retval;
} else if (retval > 0) {
- trace_rpm_usage_rcuidle(dev, rpmflags);
+ trace_rpm_usage(dev, rpmflags);
return 0;
}
}
@@ -1119,7 +1119,7 @@ int __pm_runtime_suspend(struct device *
if (retval < 0) {
return retval;
} else if (retval > 0) {
- trace_rpm_usage_rcuidle(dev, rpmflags);
+ trace_rpm_usage(dev, rpmflags);
return 0;
}
}
@@ -1202,7 +1202,7 @@ int pm_runtime_get_if_active(struct devi
} else {
retval = atomic_inc_not_zero(&dev->power.usage_count);
}
- trace_rpm_usage_rcuidle(dev, 0);
+ trace_rpm_usage(dev, 0);
spin_unlock_irqrestore(&dev->power.lock, flags);

return retval;
@@ -1566,7 +1566,7 @@ void pm_runtime_allow(struct device *dev
if (ret == 0)
rpm_idle(dev, RPM_AUTO | RPM_ASYNC);
else if (ret > 0)
- trace_rpm_usage_rcuidle(dev, RPM_AUTO | RPM_ASYNC);
+ trace_rpm_usage(dev, RPM_AUTO | RPM_ASYNC);

out:
spin_unlock_irq(&dev->power.lock);
@@ -1635,7 +1635,7 @@ static void update_autosuspend(struct de
atomic_inc(&dev->power.usage_count);
rpm_resume(dev, 0);
} else {
- trace_rpm_usage_rcuidle(dev, 0);
+ trace_rpm_usage(dev, 0);
}
}



Peter Zijlstra

unread,
Sep 19, 2022, 6:17:47 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com, Marc Zyngier
Ever since commit d3afc7f12987 ("arm64: Allow IPIs to be handled as
normal interrupts") this function is called in regular IRQ context.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Acked-by: Mark Rutland <mark.r...@arm.com>
Acked-by: Marc Zyngier <m...@kernel.org>
---
arch/arm64/kernel/smp.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -865,7 +865,7 @@ static void do_handle_IPI(int ipinr)
unsigned int cpu = smp_processor_id();

if ((unsigned)ipinr < NR_IPI)
- trace_ipi_entry_rcuidle(ipi_types[ipinr]);
+ trace_ipi_entry(ipi_types[ipinr]);

switch (ipinr) {
case IPI_RESCHEDULE:
@@ -914,7 +914,7 @@ static void do_handle_IPI(int ipinr)
}

if ((unsigned)ipinr < NR_IPI)
- trace_ipi_exit_rcuidle(ipi_types[ipinr]);
+ trace_ipi_exit(ipi_types[ipinr]);
}

static irqreturn_t ipi_handler(int irq, void *data)


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:49 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: intel_idle_s2idle+0x6e: call to __monitor.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_irq+0x8c: call to __monitor.constprop.0() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle+0x73: call to __monitor.constprop.0() leaves .noinstr.text section

vmlinux.o: warning: objtool: mwait_idle+0x88: call to clflush() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/include/asm/mwait.h | 12 ++++++------
arch/x86/include/asm/special_insns.h | 2 +-
2 files changed, 7 insertions(+), 7 deletions(-)

--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -25,7 +25,7 @@
#define TPAUSE_C01_STATE 1
#define TPAUSE_C02_STATE 0

-static inline void __monitor(const void *eax, unsigned long ecx,
+static __always_inline void __monitor(const void *eax, unsigned long ecx,
unsigned long edx)
{
/* "monitor %eax, %ecx, %edx;" */
@@ -33,7 +33,7 @@ static inline void __monitor(const void
:: "a" (eax), "c" (ecx), "d"(edx));
}

-static inline void __monitorx(const void *eax, unsigned long ecx,
+static __always_inline void __monitorx(const void *eax, unsigned long ecx,
unsigned long edx)
{
/* "monitorx %eax, %ecx, %edx;" */
@@ -41,7 +41,7 @@ static inline void __monitorx(const void
:: "a" (eax), "c" (ecx), "d"(edx));
}

-static inline void __mwait(unsigned long eax, unsigned long ecx)
+static __always_inline void __mwait(unsigned long eax, unsigned long ecx)
{
mds_idle_clear_cpu_buffers();

@@ -76,8 +76,8 @@ static inline void __mwait(unsigned long
* EAX (logical) address to monitor
* ECX #GP if not zero
*/
-static inline void __mwaitx(unsigned long eax, unsigned long ebx,
- unsigned long ecx)
+static __always_inline void __mwaitx(unsigned long eax, unsigned long ebx,
+ unsigned long ecx)
{
/* No MDS buffer clear as this is AMD/HYGON only */

@@ -86,7 +86,7 @@ static inline void __mwaitx(unsigned lon
:: "a" (eax), "b" (ebx), "c" (ecx));
}

-static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
+static __always_inline void __sti_mwait(unsigned long eax, unsigned long ecx)
{
mds_idle_clear_cpu_buffers();
/* "mwait %eax, %ecx;" */
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -196,7 +196,7 @@ static inline void load_gs_index(unsigne

#endif /* CONFIG_PARAVIRT_XXL */

-static inline void clflush(volatile void *__p)
+static __always_inline void clflush(volatile void *__p)
{
asm volatile("clflush %0" : "+m" (*(volatile char __force *)__p));
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:50 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
OMAP3 uses full SoC suspend modes as idle states, as such it needs the
whole power-domain and clock-domain code from the idle path.

All that code is not suitable to run with RCU disabled, as such push
RCU-idle deeper still.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Tony Lindgren <to...@atomide.com>
Tested-by: Tony Lindgren <to...@atomide.com>
---
arch/arm/mach-omap2/cpuidle34xx.c | 4 +---
arch/arm/mach-omap2/pm.h | 2 +-
arch/arm/mach-omap2/pm34xx.c | 12 ++++++++++--
3 files changed, 12 insertions(+), 6 deletions(-)

--- a/arch/arm/mach-omap2/cpuidle34xx.c
+++ b/arch/arm/mach-omap2/cpuidle34xx.c
@@ -133,9 +133,7 @@ static int omap3_enter_idle(struct cpuid
}

/* Execute ARM wfi */
- ct_cpuidle_enter();
- omap_sram_idle();
- ct_cpuidle_exit();
+ omap_sram_idle(true);

/*
* Call idle CPU PM enter notifier chain to restore
--- a/arch/arm/mach-omap2/pm.h
+++ b/arch/arm/mach-omap2/pm.h
@@ -29,7 +29,7 @@ static inline int omap4_idle_init(void)

extern void *omap3_secure_ram_storage;
extern void omap3_pm_off_mode_enable(int);
-extern void omap_sram_idle(void);
+extern void omap_sram_idle(bool rcuidle);
extern int omap_pm_clkdms_setup(struct clockdomain *clkdm, void *unused);

#if defined(CONFIG_PM_OPP)
--- a/arch/arm/mach-omap2/pm34xx.c
+++ b/arch/arm/mach-omap2/pm34xx.c
@@ -26,6 +26,7 @@
#include <linux/delay.h>
#include <linux/slab.h>
#include <linux/of.h>
+#include <linux/cpuidle.h>

#include <trace/events/power.h>

@@ -174,7 +175,7 @@ static int omap34xx_do_sram_idle(unsigne
return 0;
}

-void omap_sram_idle(void)
+void omap_sram_idle(bool rcuidle)
{
/* Variable to tell what needs to be saved and restored
* in omap_sram_idle*/
@@ -254,11 +255,18 @@ void omap_sram_idle(void)
*/
if (save_state)
omap34xx_save_context(omap3_arm_context);
+
+ if (rcuidle)
+ ct_cpuidle_enter();
+
if (save_state == 1 || save_state == 3)
cpu_suspend(save_state, omap34xx_do_sram_idle);
else
omap34xx_do_sram_idle(save_state);

+ if (rcuidle)
+ ct_cpuidle_exit();
+
/* Restore normal SDRC POWER settings */
if (cpu_is_omap3430() && omap_rev() >= OMAP3430_REV_ES3_0 &&
(omap_type() == OMAP2_DEVICE_TYPE_EMU ||
@@ -316,7 +324,7 @@ static int omap3_pm_suspend(void)

omap3_intc_suspend();

- omap_sram_idle();
+ omap_sram_idle(false);

restore:
/* Restore next_pwrsts */


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:51 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
arch_cpu_idle() is a very simple idle interface and exposes only a
single idle state and is expected to not require RCU and not do any
tracing/instrumentation.

As such, omap2_pm_idle() is not a valid implementation. Replace it
with a simple (shallow) omap2_do_wfi() call.

Omap2 doesn't have a cpuidle driver; but adding one would be the
recourse to (re)gain the other idle states.

Suggested-by: Tony Lindgren <to...@atomide.com>
Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/arm/mach-omap2/pm24xx.c | 51 +------------------------------------------
1 file changed, 2 insertions(+), 49 deletions(-)

--- a/arch/arm/mach-omap2/pm24xx.c
+++ b/arch/arm/mach-omap2/pm24xx.c
@@ -116,50 +116,12 @@ static int omap2_enter_full_retention(vo

static int sti_console_enabled;

-static int omap2_allow_mpu_retention(void)
-{
- if (!omap2xxx_cm_mpu_retention_allowed())
- return 0;
- if (sti_console_enabled)
- return 0;
-
- return 1;
-}
-
-static void omap2_enter_mpu_retention(void)
+static void omap2_do_wfi(void)
{
const int zero = 0;

- /* The peripherals seem not to be able to wake up the MPU when
- * it is in retention mode. */
- if (omap2_allow_mpu_retention()) {
- /* REVISIT: These write to reserved bits? */
- omap_prm_clear_mod_irqs(CORE_MOD, PM_WKST1, ~0);
- omap_prm_clear_mod_irqs(CORE_MOD, OMAP24XX_PM_WKST2, ~0);
- omap_prm_clear_mod_irqs(WKUP_MOD, PM_WKST, ~0);
-
- /* Try to enter MPU retention */
- pwrdm_set_next_pwrst(mpu_pwrdm, PWRDM_POWER_RET);
-
- } else {
- /* Block MPU retention */
- pwrdm_set_next_pwrst(mpu_pwrdm, PWRDM_POWER_ON);
- }
-
/* WFI */
asm("mcr p15, 0, %0, c7, c0, 4" : : "r" (zero) : "memory", "cc");
-
- pwrdm_set_next_pwrst(mpu_pwrdm, PWRDM_POWER_ON);
-}
-
-static int omap2_can_sleep(void)
-{
- if (omap2xxx_cm_fclks_active())
- return 0;
- if (__clk_is_enabled(osc_ck))
- return 0;
-
- return 1;
}

static void omap2_pm_idle(void)
@@ -169,16 +131,7 @@ static void omap2_pm_idle(void)
if (omap_irq_pending())
return;

- error = cpu_cluster_pm_enter();
- if (error || !omap2_can_sleep()) {
- omap2_enter_mpu_retention();
- goto out_cpu_cluster_pm;
- }
-
- omap2_enter_full_retention();
-
-out_cpu_cluster_pm:
- cpu_cluster_pm_exit();
+ omap2_do_wfi();
}

static void __init prcm_setup_regs(void)


Peter Zijlstra

unread,
Sep 19, 2022, 6:17:52 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
clang-14 allyesconfig gives:

vmlinux.o: warning: objtool: emulator_cmpxchg_emulated+0x705: call to __ubsan_handle_load_invalid_value() with UACCESS enabled
vmlinux.o: warning: objtool: paging64_update_accessed_dirty_bits+0x39e: call to __ubsan_handle_load_invalid_value() with UACCESS enabled
vmlinux.o: warning: objtool: paging32_update_accessed_dirty_bits+0x390: call to __ubsan_handle_load_invalid_value() with UACCESS enabled
vmlinux.o: warning: objtool: ept_update_accessed_dirty_bits+0x43f: call to __ubsan_handle_load_invalid_value() with UACCESS enabled

Add the required eflags save/restore and whitelist the thing.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
lib/ubsan.c | 5 ++++-
tools/objtool/check.c | 1 +
2 files changed, 5 insertions(+), 1 deletion(-)

--- a/lib/ubsan.c
+++ b/lib/ubsan.c
@@ -340,9 +340,10 @@ void __ubsan_handle_load_invalid_value(v
{
struct invalid_value_data *data = _data;
char val_str[VALUE_LENGTH];
+ unsigned long ua_flags = user_access_save();

if (suppress_report(&data->location))
- return;
+ goto out;

ubsan_prologue(&data->location, "invalid-load");

@@ -352,6 +353,8 @@ void __ubsan_handle_load_invalid_value(v
val_str, data->type->type_name);

ubsan_epilogue();
+out:
+ user_access_restore(ua_flags);
}
EXPORT_SYMBOL(__ubsan_handle_load_invalid_value);

--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -1068,6 +1068,7 @@ static const char *uaccess_safe_builtin[
"__ubsan_handle_type_mismatch",
"__ubsan_handle_type_mismatch_v1",
"__ubsan_handle_shift_out_of_bounds",
+ "__ubsan_handle_load_invalid_value",
/* misc */
"csum_partial_copy_generic",
"copy_mc_fragile",


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then temporarily enable it
again, some *four* times, before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Tony Lindgren <to...@atomide.com>
Tested-by: Tony Lindgren <to...@atomide.com>
---
arch/arm/mach-omap2/cpuidle44xx.c | 29 ++++++++++++++++++-----------
1 file changed, 18 insertions(+), 11 deletions(-)

--- a/arch/arm/mach-omap2/cpuidle44xx.c
+++ b/arch/arm/mach-omap2/cpuidle44xx.c
@@ -105,7 +105,9 @@ static int omap_enter_idle_smp(struct cp
}
raw_spin_unlock_irqrestore(&mpu_lock, flag);

+ ct_idle_enter();
omap4_enter_lowpower(dev->cpu, cx->cpu_state);
+ ct_idle_exit();

raw_spin_lock_irqsave(&mpu_lock, flag);
if (cx->mpu_state_vote == num_online_cpus())
@@ -151,10 +153,10 @@ static int omap_enter_idle_coupled(struc
(cx->mpu_logic_state == PWRDM_POWER_OFF);

/* Enter broadcast mode for periodic timers */
- RCU_NONIDLE(tick_broadcast_enable());
+ tick_broadcast_enable();

/* Enter broadcast mode for one-shot timers */
- RCU_NONIDLE(tick_broadcast_enter());
+ tick_broadcast_enter();

/*
* Call idle CPU PM enter notifier chain so that
@@ -166,7 +168,7 @@ static int omap_enter_idle_coupled(struc

if (dev->cpu == 0) {
pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
- RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state));
+ omap_set_pwrdm_state(mpu_pd, cx->mpu_state);

/*
* Call idle CPU cluster PM enter notifier chain
@@ -178,14 +180,16 @@ static int omap_enter_idle_coupled(struc
index = 0;
cx = state_ptr + index;
pwrdm_set_logic_retst(mpu_pd, cx->mpu_logic_state);
- RCU_NONIDLE(omap_set_pwrdm_state(mpu_pd, cx->mpu_state));
+ omap_set_pwrdm_state(mpu_pd, cx->mpu_state);
mpuss_can_lose_context = 0;
}
}
}

+ ct_idle_enter();
omap4_enter_lowpower(dev->cpu, cx->cpu_state);
cpu_done[dev->cpu] = true;
+ ct_idle_exit();

/* Wakeup CPU1 only if it is not offlined */
if (dev->cpu == 0 && cpumask_test_cpu(1, cpu_online_mask)) {
@@ -194,9 +198,9 @@ static int omap_enter_idle_coupled(struc
mpuss_can_lose_context)
gic_dist_disable();

- RCU_NONIDLE(clkdm_deny_idle(cpu_clkdm[1]));
- RCU_NONIDLE(omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON));
- RCU_NONIDLE(clkdm_allow_idle(cpu_clkdm[1]));
+ clkdm_deny_idle(cpu_clkdm[1]);
+ omap_set_pwrdm_state(cpu_pd[1], PWRDM_POWER_ON);
+ clkdm_allow_idle(cpu_clkdm[1]);

if (IS_PM44XX_ERRATUM(PM_OMAP4_ROM_SMP_BOOT_ERRATUM_GICD) &&
mpuss_can_lose_context) {
@@ -222,7 +226,7 @@ static int omap_enter_idle_coupled(struc
cpu_pm_exit();

cpu_pm_out:
- RCU_NONIDLE(tick_broadcast_exit());
+ tick_broadcast_exit();

fail:
cpuidle_coupled_parallel_barrier(dev, &abort_barrier);
@@ -247,7 +251,8 @@ static struct cpuidle_driver omap4_idle_
/* C2 - CPU0 OFF + CPU1 OFF + MPU CSWR */
.exit_latency = 328 + 440,
.target_residency = 960,
- .flags = CPUIDLE_FLAG_COUPLED,
+ .flags = CPUIDLE_FLAG_COUPLED |
+ CPUIDLE_FLAG_RCU_IDLE,
.enter = omap_enter_idle_coupled,
.name = "C2",
.desc = "CPUx OFF, MPUSS CSWR",
@@ -256,7 +261,8 @@ static struct cpuidle_driver omap4_idle_
/* C3 - CPU0 OFF + CPU1 OFF + MPU OSWR */
.exit_latency = 460 + 518,
.target_residency = 1100,
- .flags = CPUIDLE_FLAG_COUPLED,
+ .flags = CPUIDLE_FLAG_COUPLED |
+ CPUIDLE_FLAG_RCU_IDLE,
.enter = omap_enter_idle_coupled,
.name = "C3",
.desc = "CPUx OFF, MPUSS OSWR",
@@ -282,7 +288,8 @@ static struct cpuidle_driver omap5_idle_
/* C2 - CPU0 RET + CPU1 RET + MPU CSWR */
.exit_latency = 48 + 60,
.target_residency = 100,
- .flags = CPUIDLE_FLAG_TIMER_STOP,
+ .flags = CPUIDLE_FLAG_TIMER_STOP |
+ CPUIDLE_FLAG_RCU_IDLE,
.enter = omap_enter_idle_smp,
.name = "C2",
.desc = "CPUx CSWR, MPUSS CSWR",


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
The problem, per commit fc98c3c8c9dc ("printk: use rcuidle console
tracepoint"), was printk usage from the cpuidle path where RCU was
already disabled.

Per the patches earlier in this series, this is no longer the case.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Reviewed-by: Sergey Senozhatsky <senoz...@chromium.org>
Acked-by: Petr Mladek <pml...@suse.com>
---
kernel/printk/printk.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

--- a/kernel/printk/printk.c
+++ b/kernel/printk/printk.c
@@ -2238,7 +2238,7 @@ static u16 printk_sprint(char *text, u16
}
}

- trace_console_rcuidle(text, text_len);
+ trace_console(text, text_len);

return text_len;
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
The PM notifiers should no longer be ran with RCU disabled (per the
previous patches), as such this hack is no longer required either.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/perf/arm_pmu.c | 11 +----------
drivers/perf/riscv_pmu_sbi.c | 8 +-------
2 files changed, 2 insertions(+), 17 deletions(-)

--- a/drivers/perf/arm_pmu.c
+++ b/drivers/perf/arm_pmu.c
@@ -762,17 +762,8 @@ static void cpu_pm_pmu_setup(struct arm_
case CPU_PM_ENTER_FAILED:
/*
* Restore and enable the counter.
- * armpmu_start() indirectly calls
- *
- * perf_event_update_userpage()
- *
- * that requires RCU read locking to be functional,
- * wrap the call within RCU_NONIDLE to make the
- * RCU subsystem aware this cpu is not idle from
- * an RCU perspective for the armpmu_start() call
- * duration.
*/
- RCU_NONIDLE(armpmu_start(event, PERF_EF_RELOAD));
+ armpmu_start(event, PERF_EF_RELOAD);
break;
default:
break;
--- a/drivers/perf/riscv_pmu_sbi.c
+++ b/drivers/perf/riscv_pmu_sbi.c
@@ -747,14 +747,8 @@ static int riscv_pm_pmu_notify(struct no
case CPU_PM_ENTER_FAILED:
/*
* Restore and enable the counter.
- *
- * Requires RCU read locking to be functional,
- * wrap the call within RCU_NONIDLE to make the
- * RCU subsystem aware this cpu is not idle from
- * an RCU perspective for the riscv_pmu_start() call
- * duration.
*/
- RCU_NONIDLE(riscv_pmu_start(event, PERF_EF_RELOAD));
+ riscv_pmu_start(event, PERF_EF_RELOAD);
break;
default:
break;


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: intel_idle_s2idle+0xd5: call to fpu_idle_fpregs() leaves .noinstr.text section
vmlinux.o: warning: objtool: intel_idle_xstate+0x11: call to fpu_idle_fpregs() leaves .noinstr.text section
vmlinux.o: warning: objtool: fpu_idle_fpregs+0x9: call to xfeatures_in_use() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/include/asm/fpu/xcr.h | 4 ++--
arch/x86/include/asm/special_insns.h | 2 +-
arch/x86/kernel/fpu/core.c | 4 ++--
3 files changed, 5 insertions(+), 5 deletions(-)

--- a/arch/x86/include/asm/fpu/xcr.h
+++ b/arch/x86/include/asm/fpu/xcr.h
@@ -5,7 +5,7 @@
#define XCR_XFEATURE_ENABLED_MASK 0x00000000
#define XCR_XFEATURE_IN_USE_MASK 0x00000001

-static inline u64 xgetbv(u32 index)
+static __always_inline u64 xgetbv(u32 index)
{
u32 eax, edx;

@@ -27,7 +27,7 @@ static inline void xsetbv(u32 index, u64
*
* Callers should check X86_FEATURE_XGETBV1.
*/
-static inline u64 xfeatures_in_use(void)
+static __always_inline u64 xfeatures_in_use(void)
{
return xgetbv(XCR_XFEATURE_IN_USE_MASK);
}
--- a/arch/x86/include/asm/special_insns.h
+++ b/arch/x86/include/asm/special_insns.h
@@ -295,7 +295,7 @@ static inline int enqcmds(void __iomem *
return 0;
}

-static inline void tile_release(void)
+static __always_inline void tile_release(void)
{
/*
* Instruction opcode for TILERELEASE; supported in binutils
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -856,12 +856,12 @@ int fpu__exception_code(struct fpu *fpu,
* Initialize register state that may prevent from entering low-power idle.
* This function will be invoked from the cpuidle driver only when needed.
*/
-void fpu_idle_fpregs(void)
+noinstr void fpu_idle_fpregs(void)
{
/* Note: AMX_TILE being enabled implies XGETBV1 support */
if (cpu_feature_enabled(X86_FEATURE_AMX_TILE) &&
(xfeatures_in_use() & XFEATURE_MASK_XTILE)) {
tile_release();
- fpregs_deactivate(&current->thread.fpu);
+ __this_cpu_write(fpu_fpregs_owner_ctx, NULL);
}
}


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Idle code is very like entry code in that RCU isn't available. As
such, add a little validation.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Acked-by: Geert Uytterhoeven <ge...@linux-m68k.org>
---
arch/alpha/kernel/vmlinux.lds.S | 1 -
arch/arc/kernel/vmlinux.lds.S | 1 -
arch/arm/include/asm/vmlinux.lds.h | 1 -
arch/arm64/kernel/vmlinux.lds.S | 1 -
arch/csky/kernel/vmlinux.lds.S | 1 -
arch/hexagon/kernel/vmlinux.lds.S | 1 -
arch/ia64/kernel/vmlinux.lds.S | 1 -
arch/loongarch/kernel/vmlinux.lds.S | 1 -
arch/m68k/kernel/vmlinux-nommu.lds | 1 -
arch/m68k/kernel/vmlinux-std.lds | 1 -
arch/m68k/kernel/vmlinux-sun3.lds | 1 -
arch/microblaze/kernel/vmlinux.lds.S | 1 -
arch/mips/kernel/vmlinux.lds.S | 1 -
arch/nios2/kernel/vmlinux.lds.S | 1 -
arch/openrisc/kernel/vmlinux.lds.S | 1 -
arch/parisc/kernel/vmlinux.lds.S | 1 -
arch/powerpc/kernel/vmlinux.lds.S | 1 -
arch/riscv/kernel/vmlinux-xip.lds.S | 1 -
arch/riscv/kernel/vmlinux.lds.S | 1 -
arch/s390/kernel/vmlinux.lds.S | 1 -
arch/sh/kernel/vmlinux.lds.S | 1 -
arch/sparc/kernel/vmlinux.lds.S | 1 -
arch/um/kernel/dyn.lds.S | 1 -
arch/um/kernel/uml.lds.S | 1 -
arch/x86/include/asm/irqflags.h | 11 ++++-------
arch/x86/include/asm/mwait.h | 2 +-
arch/x86/kernel/vmlinux.lds.S | 1 -
arch/xtensa/kernel/vmlinux.lds.S | 1 -
include/asm-generic/vmlinux.lds.h | 9 +++------
include/linux/compiler_types.h | 8 ++++++--
include/linux/cpu.h | 3 ---
tools/objtool/check.c | 13 +++++++++++++
32 files changed, 27 insertions(+), 45 deletions(-)

--- a/arch/alpha/kernel/vmlinux.lds.S
+++ b/arch/alpha/kernel/vmlinux.lds.S
@@ -27,7 +27,6 @@ SECTIONS
HEAD_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
*(.fixup)
*(.gnu.warning)
--- a/arch/arc/kernel/vmlinux.lds.S
+++ b/arch/arc/kernel/vmlinux.lds.S
@@ -85,7 +85,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/arm/include/asm/vmlinux.lds.h
+++ b/arch/arm/include/asm/vmlinux.lds.h
@@ -96,7 +96,6 @@
SOFTIRQENTRY_TEXT \
TEXT_TEXT \
SCHED_TEXT \
- CPUIDLE_TEXT \
LOCK_TEXT \
KPROBES_TEXT \
ARM_STUBS_TEXT \
--- a/arch/arm64/kernel/vmlinux.lds.S
+++ b/arch/arm64/kernel/vmlinux.lds.S
@@ -163,7 +163,6 @@ SECTIONS
ENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
HYPERVISOR_TEXT
--- a/arch/csky/kernel/vmlinux.lds.S
+++ b/arch/csky/kernel/vmlinux.lds.S
@@ -38,7 +38,6 @@ SECTIONS
SOFTIRQENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
*(.fixup)
--- a/arch/hexagon/kernel/vmlinux.lds.S
+++ b/arch/hexagon/kernel/vmlinux.lds.S
@@ -41,7 +41,6 @@ SECTIONS
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
*(.fixup)
--- a/arch/ia64/kernel/vmlinux.lds.S
+++ b/arch/ia64/kernel/vmlinux.lds.S
@@ -51,7 +51,6 @@ SECTIONS {
__end_ivt_text = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/loongarch/kernel/vmlinux.lds.S
+++ b/arch/loongarch/kernel/vmlinux.lds.S
@@ -41,7 +41,6 @@ SECTIONS
.text : {
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/m68k/kernel/vmlinux-nommu.lds
+++ b/arch/m68k/kernel/vmlinux-nommu.lds
@@ -48,7 +48,6 @@ SECTIONS {
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
*(.fixup)
. = ALIGN(16);
--- a/arch/m68k/kernel/vmlinux-std.lds
+++ b/arch/m68k/kernel/vmlinux-std.lds
@@ -19,7 +19,6 @@ SECTIONS
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
*(.fixup)
*(.gnu.warning)
--- a/arch/m68k/kernel/vmlinux-sun3.lds
+++ b/arch/m68k/kernel/vmlinux-sun3.lds
@@ -19,7 +19,6 @@ SECTIONS
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
*(.fixup)
*(.gnu.warning)
--- a/arch/microblaze/kernel/vmlinux.lds.S
+++ b/arch/microblaze/kernel/vmlinux.lds.S
@@ -36,7 +36,6 @@ SECTIONS {
EXIT_TEXT
EXIT_CALL
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/mips/kernel/vmlinux.lds.S
+++ b/arch/mips/kernel/vmlinux.lds.S
@@ -61,7 +61,6 @@ SECTIONS
.text : {
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/nios2/kernel/vmlinux.lds.S
+++ b/arch/nios2/kernel/vmlinux.lds.S
@@ -24,7 +24,6 @@ SECTIONS
.text : {
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
--- a/arch/openrisc/kernel/vmlinux.lds.S
+++ b/arch/openrisc/kernel/vmlinux.lds.S
@@ -52,7 +52,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/parisc/kernel/vmlinux.lds.S
+++ b/arch/parisc/kernel/vmlinux.lds.S
@@ -86,7 +86,6 @@ SECTIONS
TEXT_TEXT
LOCK_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
--- a/arch/powerpc/kernel/vmlinux.lds.S
+++ b/arch/powerpc/kernel/vmlinux.lds.S
@@ -107,7 +107,6 @@ SECTIONS
#endif
NOINSTR_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/riscv/kernel/vmlinux-xip.lds.S
+++ b/arch/riscv/kernel/vmlinux-xip.lds.S
@@ -39,7 +39,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
ENTRY_TEXT
--- a/arch/riscv/kernel/vmlinux.lds.S
+++ b/arch/riscv/kernel/vmlinux.lds.S
@@ -42,7 +42,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
ENTRY_TEXT
--- a/arch/s390/kernel/vmlinux.lds.S
+++ b/arch/s390/kernel/vmlinux.lds.S
@@ -42,7 +42,6 @@ SECTIONS
HEAD_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/sh/kernel/vmlinux.lds.S
+++ b/arch/sh/kernel/vmlinux.lds.S
@@ -29,7 +29,6 @@ SECTIONS
HEAD_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/sparc/kernel/vmlinux.lds.S
+++ b/arch/sparc/kernel/vmlinux.lds.S
@@ -50,7 +50,6 @@ SECTIONS
HEAD_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
IRQENTRY_TEXT
--- a/arch/um/kernel/dyn.lds.S
+++ b/arch/um/kernel/dyn.lds.S
@@ -74,7 +74,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
--- a/arch/um/kernel/uml.lds.S
+++ b/arch/um/kernel/uml.lds.S
@@ -35,7 +35,6 @@ SECTIONS
_stext = .;
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
IRQENTRY_TEXT
SOFTIRQENTRY_TEXT
--- a/arch/x86/include/asm/irqflags.h
+++ b/arch/x86/include/asm/irqflags.h
@@ -8,9 +8,6 @@

#include <asm/nospec-branch.h>

-/* Provide __cpuidle; we can't safely include <linux/cpu.h> */
-#define __cpuidle __section(".cpuidle.text")
-
/*
* Interrupt control:
*/
@@ -45,13 +42,13 @@ static __always_inline void native_irq_e
asm volatile("sti": : :"memory");
}

-static inline __cpuidle void native_safe_halt(void)
+static __always_inline void native_safe_halt(void)
{
mds_idle_clear_cpu_buffers();
asm volatile("sti; hlt": : :"memory");
}

-static inline __cpuidle void native_halt(void)
+static __always_inline void native_halt(void)
{
mds_idle_clear_cpu_buffers();
asm volatile("hlt": : :"memory");
@@ -84,7 +81,7 @@ static __always_inline void arch_local_i
* Used in the idle loop; sti takes one instruction cycle
* to complete:
*/
-static inline __cpuidle void arch_safe_halt(void)
+static __always_inline void arch_safe_halt(void)
{
native_safe_halt();
}
@@ -93,7 +90,7 @@ static inline __cpuidle void arch_safe_h
* Used when interrupts are already enabled or to
* shutdown the processor:
*/
-static inline __cpuidle void halt(void)
+static __always_inline void halt(void)
{
native_halt();
}
--- a/arch/x86/include/asm/mwait.h
+++ b/arch/x86/include/asm/mwait.h
@@ -104,7 +104,7 @@ static inline void __sti_mwait(unsigned
* New with Core Duo processors, MWAIT can take some hints based on CPU
* capability.
*/
-static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx)
+static __always_inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx)
{
if (static_cpu_has_bug(X86_BUG_MONITOR) || !current_set_polling_and_test()) {
if (static_cpu_has_bug(X86_BUG_CLFLUSH_MONITOR)) {
--- a/arch/x86/kernel/vmlinux.lds.S
+++ b/arch/x86/kernel/vmlinux.lds.S
@@ -129,7 +129,6 @@ SECTIONS
HEAD_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
KPROBES_TEXT
ALIGN_ENTRY_TEXT_BEGIN
--- a/arch/xtensa/kernel/vmlinux.lds.S
+++ b/arch/xtensa/kernel/vmlinux.lds.S
@@ -125,7 +125,6 @@ SECTIONS
ENTRY_TEXT
TEXT_TEXT
SCHED_TEXT
- CPUIDLE_TEXT
LOCK_TEXT
*(.fixup)
}
--- a/include/asm-generic/vmlinux.lds.h
+++ b/include/asm-generic/vmlinux.lds.h
@@ -559,6 +559,9 @@
ALIGN_FUNCTION(); \
__noinstr_text_start = .; \
*(.noinstr.text) \
+ __cpuidle_text_start = .; \
+ *(.cpuidle.text) \
+ __cpuidle_text_end = .; \
__noinstr_text_end = .;

/*
@@ -600,12 +603,6 @@
*(.spinlock.text) \
__lock_text_end = .;

-#define CPUIDLE_TEXT \
- ALIGN_FUNCTION(); \
- __cpuidle_text_start = .; \
- *(.cpuidle.text) \
- __cpuidle_text_end = .;
-
#define KPROBES_TEXT \
ALIGN_FUNCTION(); \
__kprobes_text_start = .; \
--- a/include/linux/compiler_types.h
+++ b/include/linux/compiler_types.h
@@ -227,10 +227,14 @@ struct ftrace_likely_data {
#endif

/* Section for code which can't be instrumented at all */
-#define noinstr \
- noinline notrace __attribute((__section__(".noinstr.text"))) \
+#define __noinstr_section(section) \
+ noinline notrace __attribute((__section__(section))) \
__no_kcsan __no_sanitize_address __no_profile __no_sanitize_coverage

+#define noinstr __noinstr_section(".noinstr.text")
+
+#define __cpuidle __noinstr_section(".cpuidle.text")
+
#endif /* __KERNEL__ */

#endif /* __ASSEMBLY__ */
--- a/include/linux/cpu.h
+++ b/include/linux/cpu.h
@@ -176,9 +176,6 @@ void __noreturn cpu_startup_entry(enum c

void cpu_idle_poll_ctrl(bool enable);

-/* Attach to any functions which should be considered cpuidle. */
-#define __cpuidle __section(".cpuidle.text")
-
bool cpu_in_idle(unsigned long pc);

void arch_cpu_idle(void);
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -377,6 +377,7 @@ static int decode_instructions(struct ob

if (!strcmp(sec->name, ".noinstr.text") ||
!strcmp(sec->name, ".entry.text") ||
+ !strcmp(sec->name, ".cpuidle.text") ||
!strncmp(sec->name, ".text.__x86.", 12))
sec->noinstr = true;

@@ -3187,6 +3188,12 @@ static inline bool noinstr_call_dest(str
return true;

/*
+ * If the symbol is a static_call trampoline, we can't tell.
+ */
+ if (func->static_call_tramp)
+ return true;
+
+ /*
* The __ubsan_handle_*() calls are like WARN(), they only happen when
* something 'BAD' happened. At the risk of taking the machine down,
* let them proceed to get the message out.
@@ -3932,6 +3939,12 @@ static int validate_noinstr_sections(str
if (sec) {
warnings += validate_section(file, sec);
warnings += validate_unwind_hints(file, sec);
+ }
+
+ sec = find_section_by_name(file->elf, ".cpuidle.text");
+ if (sec) {
+ warnings += validate_section(file, sec);
+ warnings += validate_unwind_hints(file, sec);
}

return warnings;


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
KASAN cannot just hijack the mem*() functions, it needs to emit
__asan_mem*() variants if it wants instrumentation (other sanitizers
already do this).

vmlinux.o: warning: objtool: sync_regs+0x24: call to memcpy() leaves .noinstr.text section
vmlinux.o: warning: objtool: vc_switch_off_ist+0xbe: call to memcpy() leaves .noinstr.text section
vmlinux.o: warning: objtool: fixup_bad_iret+0x36: call to memset() leaves .noinstr.text section
vmlinux.o: warning: objtool: __sev_get_ghcb+0xa0: call to memcpy() leaves .noinstr.text section
vmlinux.o: warning: objtool: __sev_put_ghcb+0x35: call to memcpy() leaves .noinstr.text section

Remove the weak aliases to ensure nobody hijacks these functions and
add them to the noinstr section.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/lib/memcpy_64.S | 5 ++---
arch/x86/lib/memmove_64.S | 4 +++-
arch/x86/lib/memset_64.S | 4 +++-
mm/kasan/kasan.h | 4 ++++
mm/kasan/shadow.c | 38 ++++++++++++++++++++++++++++++++++++++
tools/objtool/check.c | 3 +++
6 files changed, 53 insertions(+), 5 deletions(-)

--- a/arch/x86/lib/memcpy_64.S
+++ b/arch/x86/lib/memcpy_64.S
@@ -7,7 +7,7 @@
#include <asm/alternative.h>
#include <asm/export.h>

-.pushsection .noinstr.text, "ax"
+.section .noinstr.text, "ax"

/*
* We build a jump to memcpy_orig by default which gets NOPped out on
@@ -42,7 +42,7 @@ SYM_FUNC_START(__memcpy)
SYM_FUNC_END(__memcpy)
EXPORT_SYMBOL(__memcpy)

-SYM_FUNC_ALIAS_WEAK(memcpy, __memcpy)
+SYM_FUNC_ALIAS(memcpy, __memcpy)
EXPORT_SYMBOL(memcpy)

/*
@@ -183,4 +183,3 @@ SYM_FUNC_START_LOCAL(memcpy_orig)
RET
SYM_FUNC_END(memcpy_orig)

-.popsection
--- a/arch/x86/lib/memmove_64.S
+++ b/arch/x86/lib/memmove_64.S
@@ -13,6 +13,8 @@

#undef memmove

+.section .noinstr.text, "ax"
+
/*
* Implement memmove(). This can handle overlap between src and dst.
*
@@ -213,5 +215,5 @@ SYM_FUNC_START(__memmove)
SYM_FUNC_END(__memmove)
EXPORT_SYMBOL(__memmove)

-SYM_FUNC_ALIAS_WEAK(memmove, __memmove)
+SYM_FUNC_ALIAS(memmove, __memmove)
EXPORT_SYMBOL(memmove)
--- a/arch/x86/lib/memset_64.S
+++ b/arch/x86/lib/memset_64.S
@@ -6,6 +6,8 @@
#include <asm/alternative.h>
#include <asm/export.h>

+.section .noinstr.text, "ax"
+
/*
* ISO C memset - set a memory block to a byte value. This function uses fast
* string to get better performance than the original function. The code is
@@ -43,7 +45,7 @@ SYM_FUNC_START(__memset)
SYM_FUNC_END(__memset)
EXPORT_SYMBOL(__memset)

-SYM_FUNC_ALIAS_WEAK(memset, __memset)
+SYM_FUNC_ALIAS(memset, __memset)
EXPORT_SYMBOL(memset)

/*
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -551,6 +551,10 @@ void __asan_set_shadow_f3(const void *ad
void __asan_set_shadow_f5(const void *addr, size_t size);
void __asan_set_shadow_f8(const void *addr, size_t size);

+void *__asan_memset(void *addr, int c, size_t len);
+void *__asan_memmove(void *dest, const void *src, size_t len);
+void *__asan_memcpy(void *dest, const void *src, size_t len);
+
void __hwasan_load1_noabort(unsigned long addr);
void __hwasan_store1_noabort(unsigned long addr);
void __hwasan_load2_noabort(unsigned long addr);
--- a/mm/kasan/shadow.c
+++ b/mm/kasan/shadow.c
@@ -38,6 +38,12 @@ bool __kasan_check_write(const volatile
}
EXPORT_SYMBOL(__kasan_check_write);

+#ifndef CONFIG_GENERIC_ENTRY
+/*
+ * CONFIG_GENERIC_ENTRY relies on compiler emitted mem*() calls to not be
+ * instrumented. KASAN enabled toolchains should emit __asan_mem*() functions
+ * for the sites they want to instrument.
+ */
#undef memset
void *memset(void *addr, int c, size_t len)
{
@@ -68,6 +74,38 @@ void *memcpy(void *dest, const void *src

return __memcpy(dest, src, len);
}
+#endif
+
+void *__asan_memset(void *addr, int c, size_t len)
+{
+ if (!kasan_check_range((unsigned long)addr, len, true, _RET_IP_))
+ return NULL;
+
+ return __memset(addr, c, len);
+}
+EXPORT_SYMBOL(__asan_memset);
+
+#ifdef __HAVE_ARCH_MEMMOVE
+void *__asan_memmove(void *dest, const void *src, size_t len)
+{
+ if (!kasan_check_range((unsigned long)src, len, false, _RET_IP_) ||
+ !kasan_check_range((unsigned long)dest, len, true, _RET_IP_))
+ return NULL;
+
+ return __memmove(dest, src, len);
+}
+EXPORT_SYMBOL(__asan_memmove);
+#endif
+
+void *__asan_memcpy(void *dest, const void *src, size_t len)
+{
+ if (!kasan_check_range((unsigned long)src, len, false, _RET_IP_) ||
+ !kasan_check_range((unsigned long)dest, len, true, _RET_IP_))
+ return NULL;
+
+ return __memcpy(dest, src, len);
+}
+EXPORT_SYMBOL(__asan_memcpy);

void kasan_poison(const void *addr, size_t size, u8 value, bool init)
{
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -956,6 +956,9 @@ static const char *uaccess_safe_builtin[
"__asan_store16_noabort",
"__kasan_check_read",
"__kasan_check_write",
+ "__asan_memset",
+ "__asan_memmove",
+ "__asan_memcpy",
/* KASAN in-line */
"__asan_report_load_n_noabort",
"__asan_report_load1_noabort",


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:08 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Doing RCU-idle outside the driver, only to then temporarily enable it
again, at least twice, before going idle is daft.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
drivers/cpuidle/cpuidle-riscv-sbi.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)

--- a/drivers/cpuidle/cpuidle-riscv-sbi.c
+++ b/drivers/cpuidle/cpuidle-riscv-sbi.c
@@ -116,12 +116,12 @@ static int __sbi_enter_domain_idle_state
return -1;

/* Do runtime PM to manage a hierarchical CPU toplogy. */
- ct_irq_enter_irqson();
if (s2idle)
dev_pm_genpd_suspend(pd_dev);
else
pm_runtime_put_sync_suspend(pd_dev);
- ct_irq_exit_irqson();
+
+ ct_idle_enter();

if (sbi_is_domain_state_available())
state = sbi_get_domain_state();
@@ -130,12 +130,12 @@ static int __sbi_enter_domain_idle_state

ret = sbi_suspend(state) ? -1 : idx;

- ct_irq_enter_irqson();
+ ct_idle_exit();
+
if (s2idle)
dev_pm_genpd_resume(pd_dev);
else
pm_runtime_get_sync(pd_dev);
- ct_irq_exit_irqson();

cpu_pm_exit();

@@ -246,6 +246,7 @@ static int sbi_dt_cpu_init_topology(stru
* of a shared state for the domain, assumes the domain states are all
* deeper states.
*/
+ drv->states[state_count - 1].flags |= CPUIDLE_FLAG_RCU_IDLE;
drv->states[state_count - 1].enter = sbi_enter_domain_idle_state;
drv->states[state_count - 1].enter_s2idle =
sbi_enter_s2idle_domain_idle_state;


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:09 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
No callers left that have already disabled RCU.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
Acked-by: Mark Rutland <mark.r...@arm.com>
---
kernel/time/tick-broadcast-hrtimer.c | 29 ++++++++++++-----------------
1 file changed, 12 insertions(+), 17 deletions(-)

--- a/kernel/time/tick-broadcast-hrtimer.c
+++ b/kernel/time/tick-broadcast-hrtimer.c
@@ -56,25 +56,20 @@ static int bc_set_next(ktime_t expires,
* hrtimer callback function is currently running, then
* hrtimer_start() cannot move it and the timer stays on the CPU on
* which it is assigned at the moment.
+ */
+ hrtimer_start(&bctimer, expires, HRTIMER_MODE_ABS_PINNED_HARD);
+ /*
+ * The core tick broadcast mode expects bc->bound_on to be set
+ * correctly to prevent a CPU which has the broadcast hrtimer
+ * armed from going deep idle.
*
- * As this can be called from idle code, the hrtimer_start()
- * invocation has to be wrapped with RCU_NONIDLE() as
- * hrtimer_start() can call into tracing.
+ * As tick_broadcast_lock is held, nothing can change the cpu
+ * base which was just established in hrtimer_start() above. So
+ * the below access is safe even without holding the hrtimer
+ * base lock.
*/
- RCU_NONIDLE( {
- hrtimer_start(&bctimer, expires, HRTIMER_MODE_ABS_PINNED_HARD);
- /*
- * The core tick broadcast mode expects bc->bound_on to be set
- * correctly to prevent a CPU which has the broadcast hrtimer
- * armed from going deep idle.
- *
- * As tick_broadcast_lock is held, nothing can change the cpu
- * base which was just established in hrtimer_start() above. So
- * the below access is safe even without holding the hrtimer
- * base lock.
- */
- bc->bound_on = bctimer.base->cpu_base->cpu;
- } );
+ bc->bound_on = bctimer.base->cpu_base->cpu;
+
return 0;
}



Peter Zijlstra

unread,
Sep 19, 2022, 6:18:09 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
vmlinux.o: warning: objtool: in_entry_stack+0x9: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: default_do_nmi+0x10: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: fpu_idle_fpregs+0x41: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: kvm_read_and_reset_apf_flags+0x1: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: lockdep_hardirqs_on+0xb0: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: lockdep_hardirqs_off+0xae: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: irqentry_nmi_enter+0x69: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: irqentry_nmi_exit+0x32: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_processor_ffh_cstate_enter+0x9: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_idle_enter+0x43: call to __this_cpu_preempt_check() leaves .noinstr.text section
vmlinux.o: warning: objtool: acpi_idle_enter_s2idle+0x45: call to __this_cpu_preempt_check() leaves .noinstr.text section

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
include/linux/percpu-defs.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

--- a/include/linux/percpu-defs.h
+++ b/include/linux/percpu-defs.h
@@ -310,7 +310,7 @@ extern void __bad_size_call_parameter(vo
#ifdef CONFIG_DEBUG_PREEMPT
extern void __this_cpu_preempt_check(const char *op);
#else
-static inline void __this_cpu_preempt_check(const char *op) { }
+static __always_inline void __this_cpu_preempt_check(const char *op) { }
#endif

#define __pcpu_size_call_return(stem, variable) \


Peter Zijlstra

unread,
Sep 19, 2022, 6:18:09 AMSep 19
to pet...@infradead.org, richard....@linaro.org, i...@jurassic.park.msu.ru, matt...@gmail.com, vgu...@kernel.org, li...@armlinux.org.uk, ulli....@googlemail.com, linus....@linaro.org, shaw...@kernel.org, Sascha Hauer, ker...@pengutronix.de, fest...@gmail.com, linu...@nxp.com, to...@atomide.com, khi...@kernel.org, catalin...@arm.com, wi...@kernel.org, guo...@kernel.org, bc...@quicinc.com, chenh...@kernel.org, ker...@xen0n.name, ge...@linux-m68k.org, sa...@sammy.net, mon...@monstr.eu, tsbo...@alpha.franken.de, ding...@kernel.org, jo...@southpole.se, stefan.kr...@saunalahti.fi, sho...@gmail.com, James.B...@hansenpartnership.com, del...@gmx.de, m...@ellerman.id.au, npi...@gmail.com, christop...@csgroup.eu, paul.w...@sifive.com, pal...@dabbelt.com, a...@eecs.berkeley.edu, h...@linux.ibm.com, g...@linux.ibm.com, agor...@linux.ibm.com, bornt...@linux.ibm.com, sv...@linux.ibm.com, ys...@users.sourceforge.jp, dal...@libc.org, da...@davemloft.net, ric...@nod.at, anton....@cambridgegreys.com, joha...@sipsolutions.net, tg...@linutronix.de, mi...@redhat.com, b...@alien8.de, dave....@linux.intel.com, x...@kernel.org, h...@zytor.com, ac...@kernel.org, mark.r...@arm.com, alexander...@linux.intel.com, jo...@kernel.org, namh...@kernel.org, jgr...@suse.com, sriv...@csail.mit.edu, amak...@vmware.com, pv-dr...@vmware.com, boris.o...@oracle.com, ch...@zankel.net, jcmv...@gmail.com, raf...@kernel.org, le...@kernel.org, pa...@ucw.cz, gre...@linuxfoundation.org, mturq...@baylibre.com, sb...@kernel.org, daniel....@linaro.org, lpier...@kernel.org, sudeep...@arm.com, agr...@kernel.org, bjorn.a...@linaro.org, konrad...@somainline.org, an...@brainfault.org, thierry...@gmail.com, jona...@nvidia.com, jacob....@linux.intel.com, ati...@atishpatra.org, Arnd Bergmann, yury....@gmail.com, andriy.s...@linux.intel.com, li...@rasmusvillemoes.dk, den...@kernel.org, t...@kernel.org, c...@linux.com, ros...@goodmis.org, pml...@suse.com, senoz...@chromium.org, john....@linutronix.de, juri....@redhat.com, vincent...@linaro.org, dietmar....@arm.com, bse...@google.com, mgo...@suse.de, bri...@redhat.com, vsch...@redhat.com, fwei...@gmail.com, ryabin...@gmail.com, gli...@google.com, andre...@gmail.com, dvy...@google.com, vincenzo...@arm.com, Andrew Morton, jpoi...@kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux-s...@lists.infradead.org, linux...@vger.kernel.org, linux...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, loon...@lists.linux.dev, linux...@lists.linux-m68k.org, linux...@vger.kernel.org, open...@lists.librecores.org, linux-...@vger.kernel.org, linuxp...@lists.ozlabs.org, linux...@lists.infradead.org, linux...@vger.kernel.org, linu...@vger.kernel.org, sparc...@vger.kernel.org, linu...@lists.infradead.org, linux-pe...@vger.kernel.org, virtual...@lists.linux-foundation.org, linux-...@linux-xtensa.org, linux...@vger.kernel.org, linu...@vger.kernel.org, linu...@vger.kernel.org, linux-...@vger.kernel.org, linux...@vger.kernel.org, linux...@vger.kernel.org, kasa...@googlegroups.com
Now that arch_cpu_idle() is expected to return with IRQs disabled,
avoid the useless STI/CLI dance.

Per the specs this is supposed to work, but nobody has yet relied up
this behaviour so broken implementations are possible.

Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org>
---
arch/x86/coco/tdx/tdcall.S | 13 -------------
arch/x86/coco/tdx/tdx.c | 23 ++++-------------------
arch/x86/include/asm/shared/tdx.h | 1 -
3 files changed, 4 insertions(+), 33 deletions(-)

--- a/arch/x86/coco/tdx/tdcall.S
+++ b/arch/x86/coco/tdx/tdcall.S
@@ -139,19 +139,6 @@ SYM_FUNC_START(__tdx_hypercall)

movl $TDVMCALL_EXPOSE_REGS_MASK, %ecx

- /*
- * For the idle loop STI needs to be called directly before the TDCALL
- * that enters idle (EXIT_REASON_HLT case). STI instruction enables
- * interrupts only one instruction later. If there is a window between
- * STI and the instruction that emulates the HALT state, there is a
- * chance for interrupts to happen in this window, which can delay the
- * HLT operation indefinitely. Since this is the not the desired
- * result, conditionally call STI before TDCALL.
- */
- testq $TDX_HCALL_ISSUE_STI, %rsi
- jz .Lskip_sti
- sti
-.Lskip_sti:
tdcall

/*
--- a/arch/x86/coco/tdx/tdx.c
+++ b/arch/x86/coco/tdx/tdx.c
@@ -169,7 +169,7 @@ static int ve_instr_len(struct ve_info *
}
}

-static u64 __cpuidle __halt(const bool irq_disabled, const bool do_sti)
+static u64 __cpuidle __halt