[PATCH 0/18] Allow different tracers to be compiled independently
Jan Kara
Hi,
currently, when one tracer is selected, most of tracepoints for other
tracers also gets pulled into the kernel. So for example it's not possible
to enable BLK_DEV_IO_TRACE without polluting slab allocation paths with
tracepoint checks (see changelog of patch 01). This patch set adds a
possibility for each set of trace points to be compile-enabled separately.
The first patch contains the necessary magic in linux/tracepoint.h. Other
patches just tell tracing framework about correspoding config options
and possibly introduce them if they did not exist before.
The patches in this patch set are actually completely independent so
they can be merged via respective subsystem trees. But changes are rather
tiny so I don't expect much conflicts...
Honza
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Jan Kara
Signed-off-by: Jan Kara <ja...@suse.cz>
---
include/trace/events/timer.h | 2 ++
kernel/trace/Kconfig | 9 +++++++++
2 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/include/trace/events/timer.h b/include/trace/events/timer.h
index 9496b96..2737598 100644
--- a/include/trace/events/timer.h
+++ b/include/trace/events/timer.h
@@ -1,5 +1,7 @@
#undef TRACE_SYSTEM
+#undef TRACE_CONFIG
#define TRACE_SYSTEM timer
+#define TRACE_CONFIG CONFIG_TIMER_TRACER
#if !defined(_TRACE_TIMER_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_TIMER_H
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index a9d7e33..787f52e 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -421,6 +421,15 @@ config MODULE_TRACER
If unsure, say N
+config TIMER_TRACER
+ bool "Trace timer actions"
+ select GENERIC_TRACER
+ help
+ Provide trace points for timer initialization, starting, expiring,
+ and canceling.
+
+ If unsure, say N
+
config WORKQUEUE_TRACER
bool "Trace workqueues"
select GENERIC_TRACER
--
1.6.4.2
Jan Kara
CC: net...@vger.kernel.org include/trace/events/napi.h | 2 ++
kernel/trace/Kconfig | 9 +++++++++
2 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/include/trace/events/napi.h b/include/trace/events/napi.h
index a8989c4..70cb4dd 100644
--- a/include/trace/events/napi.h
+++ b/include/trace/events/napi.h
@@ -2,6 +2,8 @@
#define _TRACE_NAPI_H_
#include <linux/netdevice.h>
+#undef TRACE_CONFIG
+#define TRACE_CONFIG CONFIG_NAPI_TRACER
#include <linux/tracepoint.h>
DECLARE_TRACE(napi_poll,
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index a02d577..88b556c 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -508,6 +508,15 @@ config SKB_TRACER
If unsure, say N.
+config NAPI_TRACER
+ bool "NAPI tracing"
+ depends on NET
+ select GENERIC_TRACER
+ help
+ Provide tracing for NAPI functions.
+
+ If unsure, say N.
+
config KPROBE_EVENT
depends on KPROBES
depends on HAVE_REGS_AND_STACK_ACCESS_API
--
1.6.4.2
Jan Kara
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/include/trace/events/kmem.h b/include/trace/events/kmem.h
index 3adca0c..1f93693 100644
--- a/include/trace/events/kmem.h
+++ b/include/trace/events/kmem.h
@@ -1,5 +1,7 @@
#undef TRACE_SYSTEM
+#undef TRACE_CONFIG
#define TRACE_SYSTEM kmem
+#define TRACE_CONFIG CONFIG_KMEMTRACE
#if !defined(_TRACE_KMEM_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_KMEM_H
--
1.6.4.2
Jan Kara
kernel/trace/Kconfig | 9 +++++++++
2 files changed, 11 insertions(+), 0 deletions(-)
diff --git a/include/trace/events/kvm.h b/include/trace/events/kvm.h
index b17d49d..c85908c 100644
--- a/include/trace/events/kvm.h
+++ b/include/trace/events/kvm.h
@@ -1,6 +1,8 @@
#if !defined(_TRACE_KVM_MAIN_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_KVM_MAIN_H
+#undef TRACE_CONFIG
+#define TRACE_CONFIG CONFIG_KVM_TRACER
#include <linux/tracepoint.h>
#undef TRACE_SYSTEM
diff --git a/kernel/trace/Kconfig b/kernel/trace/Kconfig
index 88b556c..abcd529 100644
--- a/kernel/trace/Kconfig
+++ b/kernel/trace/Kconfig
@@ -517,6 +517,15 @@ config NAPI_TRACER
If unsure, say N.
+config KVM_TRACER
+ bool "KVM tracing"
+ depends on KVM
+ select GENERIC_TRACER
+ help
+ Trace lots of various events inside KVM.
+
+ If unsure, say N.
+
config KPROBE_EVENT
depends on KPROBES
depends on HAVE_REGS_AND_STACK_ACCESS_API
--
1.6.4.2
--
Jan Kara
1 files changed, 2 insertions(+), 0 deletions(-)
diff --git a/include/trace/events/block.h b/include/trace/events/block.h
index 5fb7273..6412165 100644
--- a/include/trace/events/block.h
+++ b/include/trace/events/block.h
@@ -1,5 +1,7 @@
#undef TRACE_SYSTEM
+#undef TRACE_CONFIG
#define TRACE_SYSTEM block
+#define TRACE_CONFIG CONFIG_BLK_DEV_IO_TRACE
#if !defined(_TRACE_BLOCK_H) || defined(TRACE_HEADER_MULTI_READ)
#define _TRACE_BLOCK_H
--
1.6.4.2
Frederic Weisbecker
>
> Hi,
>
> currently, when one tracer is selected, most of tracepoints for other
> tracers also gets pulled into the kernel. So for example it's not possible
> to enable BLK_DEV_IO_TRACE without polluting slab allocation paths with
> tracepoint checks (see changelog of patch 01). This patch set adds a
> possibility for each set of trace points to be compile-enabled separately.
> The first patch contains the necessary magic in linux/tracepoint.h. Other
> patches just tell tracing framework about correspoding config options
> and possibly introduce them if they did not exist before.
> The patches in this patch set are actually completely independent so
> they can be merged via respective subsystem trees. But changes are rather
> tiny so I don't expect much conflicts...
>
> Honza
(Adding more people in Cc)
I don't know. Yeah this first looks like a good idea but once
CONFIG_EVENT_TRACING is enabled, each tracepoint is a lightweight
thing and induce a tiny overhead, probably hard to notice, and
this is going to be even more the case after the jmp label
optimization patches.
I liked the fact we had a general tracing kernel once the above
config is selected. And we don't bother telling people that to
use tool X you need CONFIG_EVENT_Y, and you need to rebuild your
kernel, etc...
I wonder if this is going to be a win.
Thomas Gleixner
> CC: Thomas Gleixner <tg...@linutronix.de>
> Signed-off-by: Jan Kara <ja...@suse.cz>
Acked-by: Thomas Gleixner <tg...@linutronix.de>
Steven, can you pick that up please ?
Steven Rostedt
> On Tue, 23 Mar 2010, Jan Kara wrote:
>
> > CC: Thomas Gleixner <tg...@linutronix.de>
> > Signed-off-by: Jan Kara <ja...@suse.cz>
>
> Acked-by: Thomas Gleixner <tg...@linutronix.de>
>
> Steven, can you pick that up please ?
I assume this is dependent on the first patch. I made some comments to
that one, and was hoping to get some better comments in the code before
doing anything with it.
-- Steve
Theodore Tso
On Mar 22, 2010, at 9:04 PM, Frederic Weisbecker wrote:
>
> I don't know. Yeah this first looks like a good idea but once
> CONFIG_EVENT_TRACING is enabled, each tracepoint is a lightweight
> thing and induce a tiny overhead, probably hard to notice, and
> this is going to be even more the case after the jmp label
> optimization patches.
>
> I liked the fact we had a general tracing kernel once the above
> config is selected. And we don't bother telling people that to
> use tool X you need CONFIG_EVENT_Y, and you need to rebuild your
> kernel, etc...
Indeed, a lot of the value of tracepoints goes away if people are compiling kernels without them and we need to get a special "tracing kernel" installed before we can debug a problem.
So I'd hope we can do the necessary optimization work so people don't feel it's necessary to enable or disable tracepoints by subsystem....
-- Ted
Ingo Molnar
* Theodore Tso <ty...@MIT.EDU> wrote:
>
> On Mar 22, 2010, at 9:04 PM, Frederic Weisbecker wrote:
> >
> > I don't know. Yeah this first looks like a good idea but once
> > CONFIG_EVENT_TRACING is enabled, each tracepoint is a lightweight thing
> > and induce a tiny overhead, probably hard to notice, and this is going to
> > be even more the case after the jmp label optimization patches.
> >
> > I liked the fact we had a general tracing kernel once the above config is
> > selected. And we don't bother telling people that to use tool X you need
> > CONFIG_EVENT_Y, and you need to rebuild your kernel, etc...
>
> Indeed, a lot of the value of tracepoints goes away if people are compiling
> kernels without them and we need to get a special "tracing kernel" installed
> before we can debug a problem.
>
> So I'd hope we can do the necessary optimization work so people don't feel
> it's necessary to enable or disable tracepoints by subsystem....
Yeah, agreed. Ultra-embedded can disable them all, but other than that i think
we should not make it too finegrained as a lot of tooling value is in the
'critical mass' that tracepoints have achieved. The power events tracepoints
are most useful when combined with scheduling events, etc.
Ingo
Mathieu Desnoyers
>
> * Theodore Tso <ty...@MIT.EDU> wrote:
>
> >
> > On Mar 22, 2010, at 9:04 PM, Frederic Weisbecker wrote:
> > >
> > > I don't know. Yeah this first looks like a good idea but once
> > > CONFIG_EVENT_TRACING is enabled, each tracepoint is a lightweight thing
> > > and induce a tiny overhead, probably hard to notice, and this is going to
> > > be even more the case after the jmp label optimization patches.
> > >
> > > I liked the fact we had a general tracing kernel once the above config is
> > > selected. And we don't bother telling people that to use tool X you need
> > > CONFIG_EVENT_Y, and you need to rebuild your kernel, etc...
> >
> > Indeed, a lot of the value of tracepoints goes away if people are compiling
> > kernels without them and we need to get a special "tracing kernel" installed
> > before we can debug a problem.
> >
> > So I'd hope we can do the necessary optimization work so people don't feel
> > it's necessary to enable or disable tracepoints by subsystem....
>
> Yeah, agreed. Ultra-embedded can disable them all, but other than that i think
> we should not make it too finegrained as a lot of tooling value is in the
> 'critical mass' that tracepoints have achieved. The power events tracepoints
> are most useful when combined with scheduling events, etc.
>
We're in complete agreement here. When I considered if it was worth it to create
such a per-tracepoint group compile-time disabling in the first place, I decided
not to do it precisely due to the added-value that comes with the availability
of system-wide tracepoints. And I think with the static jump patching, we are
now at a point where the overhead is stunningly low.
Now, space-wise, the one thing I would consider appropriate as a compromise for
small embedded systems would be to allow the TRACE_EVENT probes to be compiled
as modules.
Thanks,
Mathieu
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
Steven Rostedt
> * Ingo Molnar (mi...@elte.hu) wrote:
> >
>
> We're in complete agreement here. When I considered if it was worth it to create
> such a per-tracepoint group compile-time disabling in the first place, I decided
> not to do it precisely due to the added-value that comes with the availability
> of system-wide tracepoints. And I think with the static jump patching, we are
> now at a point where the overhead is stunningly low.
>
> Now, space-wise, the one thing I would consider appropriate as a compromise for
> small embedded systems would be to allow the TRACE_EVENT probes to be compiled
> as modules.
Yeah, I agree here. Hmm, I wonder if we could automate this. That is, we
could now, create a module per trace system if it is so configured. It
would move the functions that create, record and output the trace points
in TRACE_EVENT() into a separate module. Then you just need to load the
modules you want to trace (grouped by system).
We can make add kobjects to /sys so that udev loads them automatically
for distros.
-- Steve
Jan Kara
> On Tue, Mar 23, 2010 at 01:32:02AM +0100, Jan Kara wrote:
> >
> > Hi,
> >
> > currently, when one tracer is selected, most of tracepoints for other
> > tracers also gets pulled into the kernel. So for example it's not possible
> > to enable BLK_DEV_IO_TRACE without polluting slab allocation paths with
> > tracepoint checks (see changelog of patch 01). This patch set adds a
> > possibility for each set of trace points to be compile-enabled separately.
> > The first patch contains the necessary magic in linux/tracepoint.h. Other
> > patches just tell tracing framework about correspoding config options
> > and possibly introduce them if they did not exist before.
> > The patches in this patch set are actually completely independent so
> > they can be merged via respective subsystem trees. But changes are rather
> > tiny so I don't expect much conflicts...
> >
> > Honza
>
> (Adding more people in Cc)
>
> I don't know. Yeah this first looks like a good idea but once
> CONFIG_EVENT_TRACING is enabled, each tracepoint is a lightweight
> thing and induce a tiny overhead, probably hard to notice, and
> this is going to be even more the case after the jmp label
> optimization patches.
wanted BLK_DEV_IO_TRACE enabled in all our distro kernels but there is
a concern that this could have some impact on performance especially
in SLAB allocator due to more icache pressure or so. This is not completely
bogus concern if you look at bloat-o-meter output. For example:
function old new delta
kmem_cache_alloc 542 768 +226
But looking at the disassembly now, I can see that the difference in
inline code is actually only ~40 bytes (on x86_64) - so that's about 7%.
Not a huge deal but still noticeable.
> I liked the fact we had a general tracing kernel once the above
> config is selected. And we don't bother telling people that to
> use tool X you need CONFIG_EVENT_Y, and you need to rebuild your
> kernel, etc...
OK, I understand this. I guess I should go and measure whether disabling
tracepoints really makes some performance difference or not.
Honza
--
Jan Kara <ja...@suse.cz>
SUSE Labs, CR