Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Merge ftrace data of host and guest
81 views
Skip to first unread message
Divya Vyas
Dec 19, 2014, 7:30:06 AM12/19/14
to
Hi,
I am adding trace data of host and guest using trace-merge script .
https://15351300108708232970.googlegroups.com/attach/66533d7b41eed0b/trace-merge.pl?part=0.1&view=1&vt=ANaJVrHKDEwmNeLGDINH0B5Ud4OIUrfQ2_2w2jo08nJujH7Oc9facKD3ckT2RbCiHkRAfnYxmZnBF7qi0SeMZr2IMsPBDNjgrYkblCRkmgSk2PryLVsGe5g
guest data
qemu-kvm-2687 [003] d...50550079203669: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079206816: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079240656: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079243467: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079256103: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079268391: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079280829: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079286028: kvm_entry: [detail]
host data
comm-3826 [000] d...49836825726903: sched_wakeup: [detail]
comm-3826 [000] d...49836832225344: sched_switch: [detail]
~
Data after merging :
/opt/trace-merge/trace-merge 18446739773557710924 host.data guest.data
hcomm-3826 [000] d...49836825726903: sched_wakeup: [detail]
hcomm-3826 [000] d...49836832225344: sched_switch: [detail]
gqemu-kvm-2687 [003] d...54850231044361: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231047508: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231081348: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231084159: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231096795: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231109083: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231121521: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231126720: kvm_entry: [detail]
I think it is not desired output . Ths script is written for diffrent
format .ftrace format is ( trace-cmd report command generates) :
version = 6
CPU 0 is empty
cpus=2
<idle>-0 [001] 4401.096833: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001ed8fc00 len=84
<idle>-0 [001] 4401.096899: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98
<idle>-0 [001] 4401.096935: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98 rc=0
<idle>-0 [001] 4401.097277: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097309: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98
<idle>-0 [001] 4401.097325: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98 rc=0
<idle>-0 [001] 4401.097570: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097592: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98
<idle>-0 [001] 4401.097607: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98 rc=0
<idle>-0 [001] 4401.097830: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097843: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98
Can you let me know I there is any other method to merge the traces or
any further changes in the script
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
I am adding trace data of host and guest using trace-merge script .
https://15351300108708232970.googlegroups.com/attach/66533d7b41eed0b/trace-merge.pl?part=0.1&view=1&vt=ANaJVrHKDEwmNeLGDINH0B5Ud4OIUrfQ2_2w2jo08nJujH7Oc9facKD3ckT2RbCiHkRAfnYxmZnBF7qi0SeMZr2IMsPBDNjgrYkblCRkmgSk2PryLVsGe5g
guest data
qemu-kvm-2687 [003] d...50550079203669: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079206816: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079240656: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079243467: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079256103: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079268391: kvm_entry: [detail]
qemu-kvm-2687 [003] d...50550079280829: kvm_exit: [detail]
qemu-kvm-2687 [003] d...50550079286028: kvm_entry: [detail]
host data
comm-3826 [000] d...49836825726903: sched_wakeup: [detail]
comm-3826 [000] d...49836832225344: sched_switch: [detail]
~
Data after merging :
/opt/trace-merge/trace-merge 18446739773557710924 host.data guest.data
hcomm-3826 [000] d...49836825726903: sched_wakeup: [detail]
hcomm-3826 [000] d...49836832225344: sched_switch: [detail]
gqemu-kvm-2687 [003] d...54850231044361: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231047508: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231081348: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231084159: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231096795: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231109083: kvm_entry: [detail]
gqemu-kvm-2687 [003] d...54850231121521: kvm_exit: [detail]
gqemu-kvm-2687 [003] d...54850231126720: kvm_entry: [detail]
I think it is not desired output . Ths script is written for diffrent
format .ftrace format is ( trace-cmd report command generates) :
version = 6
CPU 0 is empty
cpus=2
<idle>-0 [001] 4401.096833: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001ed8fc00 len=84
<idle>-0 [001] 4401.096899: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98
<idle>-0 [001] 4401.096935: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98 rc=0
<idle>-0 [001] 4401.097277: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097309: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98
<idle>-0 [001] 4401.097325: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98 rc=0
<idle>-0 [001] 4401.097570: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097592: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98
<idle>-0 [001] 4401.097607: net_dev_xmit:
dev=eth0 skbaddr=0xffff88001a9fe000 len=98 rc=0
<idle>-0 [001] 4401.097830: netif_receive_skb:
dev=eth0 skbaddr=0xffff88001a9fe800 len=84
<idle>-0 [001] 4401.097843: net_dev_queue:
dev=eth0 skbaddr=0xffff88001a9fea00 len=98
Can you let me know I there is any other method to merge the traces or
any further changes in the script
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Steven Rostedt
Dec 20, 2014, 12:20:05 AM12/20/14
to
On Fri, Dec 19, 2014 at 05:52:52PM +0530, Divya Vyas wrote:
> Hi,
Hi, it would be helpful if you Cc'd me. I would have responded quicker. Luckily,
someone noticed this email and told me about it.
>
> I think it is not desired output . Ths script is written for diffrent
> format .ftrace format is ( trace-cmd report command generates) :
I'm actually working on trace-cmd this week and hopefully I'll be able to release
2.5 by x-mas.
I believe that Masami is going to start continuing work on the virt guest
tracer for trace-cmd.
Also, for now, for a crud way of merging, if both the gettimeofday clocks
of the host and guest are synced, you could do:
guest> trace-cmd record -o guest.dat --date ...
host> trace-cmd record -o host.dat --date ...
and then run:
trace-cmd report -i guest.dat -i host.dat
and trace-cmd will merge the two. The --date will add a way to sync the trace
timestamps to a wall clock and that will be used to try to interleave the
events of both the data files.
-- Steve
> Hi,
Hi, it would be helpful if you Cc'd me. I would have responded quicker. Luckily,
someone noticed this email and told me about it.
>
> I think it is not desired output . Ths script is written for diffrent
> format .ftrace format is ( trace-cmd report command generates) :
2.5 by x-mas.
I believe that Masami is going to start continuing work on the virt guest
tracer for trace-cmd.
Also, for now, for a crud way of merging, if both the gettimeofday clocks
of the host and guest are synced, you could do:
guest> trace-cmd record -o guest.dat --date ...
host> trace-cmd record -o host.dat --date ...
and then run:
trace-cmd report -i guest.dat -i host.dat
and trace-cmd will merge the two. The --date will add a way to sync the trace
timestamps to a wall clock and that will be used to try to interleave the
events of both the data files.
-- Steve
Divya Vyas
Dec 22, 2014, 2:10:05 AM12/22/14
to
Hi ,
One more thing I want to add that Yoshihiro YUNOMAE's trace-merge
script does this time synchronization. We have to calculate guest
offset and then subtract with host time .
But it is not applying on current trace-cmd report output format and
not giving desired output .
Thanks,
Divya Vyas
On Mon, Dec 22, 2014 at 12:34 PM, Divya Vyas <edivy...@gmail.com> wrote:
> Hi Steven,
>
> I am sorry but I am very new to trace-cmd , that's why I didn't knew
> what you are working on it.
>
> I will explain you whole scenario , Actually I have multiple guests
> and a host and I want to see all traces on a single file with
> trace-cmd report command and may be kernelshark also .
>
> trace-cmd report -i guest.dat -i host.dat command is giving me all
> traces is single file but the timestamp is different for guest . So
> the challenge for me is to sync the timestamp of guest and host . Can
> you advise me something in this ? How can I sync gettimeofday clocks
> of guest and host ?
>
> and kernelshark doesnt take trace-cmd report input , I think it takes
> guest.dat and host.dat .
> Is there any other viewer available for ftrace ?
One more thing I want to add that Yoshihiro YUNOMAE's trace-merge
script does this time synchronization. We have to calculate guest
offset and then subtract with host time .
But it is not applying on current trace-cmd report output format and
not giving desired output .
Thanks,
Divya Vyas
On Mon, Dec 22, 2014 at 12:34 PM, Divya Vyas <edivy...@gmail.com> wrote:
> Hi Steven,
>
> I am sorry but I am very new to trace-cmd , that's why I didn't knew
> what you are working on it.
>
> I will explain you whole scenario , Actually I have multiple guests
> and a host and I want to see all traces on a single file with
> trace-cmd report command and may be kernelshark also .
>
> trace-cmd report -i guest.dat -i host.dat command is giving me all
> traces is single file but the timestamp is different for guest . So
> the challenge for me is to sync the timestamp of guest and host . Can
> you advise me something in this ? How can I sync gettimeofday clocks
> of guest and host ?
>
> and kernelshark doesnt take trace-cmd report input , I think it takes
> guest.dat and host.dat .
> Is there any other viewer available for ftrace ?
Divya Vyas
Dec 22, 2014, 2:10:05 AM12/22/14
to
Hi Steven,
I am sorry but I am very new to trace-cmd , that's why I didn't knew
what you are working on it.
I will explain you whole scenario , Actually I have multiple guests
and a host and I want to see all traces on a single file with
trace-cmd report command and may be kernelshark also .
trace-cmd report -i guest.dat -i host.dat command is giving me all
traces is single file but the timestamp is different for guest . So
the challenge for me is to sync the timestamp of guest and host . Can
you advise me something in this ? How can I sync gettimeofday clocks
of guest and host ?
and kernelshark doesnt take trace-cmd report input , I think it takes
guest.dat and host.dat .
Is there any other viewer available for ftrace ?
On Sat, Dec 20, 2014 at 10:40 AM, Steven Rostedt <ros...@goodmis.org> wrote:
I am sorry but I am very new to trace-cmd , that's why I didn't knew
what you are working on it.
I will explain you whole scenario , Actually I have multiple guests
and a host and I want to see all traces on a single file with
trace-cmd report command and may be kernelshark also .
trace-cmd report -i guest.dat -i host.dat command is giving me all
traces is single file but the timestamp is different for guest . So
the challenge for me is to sync the timestamp of guest and host . Can
you advise me something in this ? How can I sync gettimeofday clocks
of guest and host ?
and kernelshark doesnt take trace-cmd report input , I think it takes
guest.dat and host.dat .
Is there any other viewer available for ftrace ?
On Sat, Dec 20, 2014 at 10:40 AM, Steven Rostedt <ros...@goodmis.org> wrote:
Masami Hiramatsu
Dec 22, 2014, 3:40:05 AM12/22/14
to
Hi,
I'm working on trace-cmd virt-server feature. This is just a basic feature
to allow trace-cmd using virtio-trace (no time synchronizing).
I'll send the updated version of the series soon.
(2014/12/22 16:07), Divya Vyas wrote:
> Hi ,
>
> One more thing I want to add that Yoshihiro YUNOMAE's trace-merge
> script does this time synchronization. We have to calculate guest
> offset and then subtract with host time .
Right, to complete this work we need trace-merge like merging feature.
I assume that we can do that by passing the offset of the timestamp
information with guest trace data (that is the simplest implementation).
However, currently trace-merge depends on x86 TSC, but some intel
processors TSC is not stable. That is current barrier of trace-merge.
Thank you,
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hi...@hitachi.com
I'm working on trace-cmd virt-server feature. This is just a basic feature
to allow trace-cmd using virtio-trace (no time synchronizing).
I'll send the updated version of the series soon.
(2014/12/22 16:07), Divya Vyas wrote:
> Hi ,
>
> One more thing I want to add that Yoshihiro YUNOMAE's trace-merge
> script does this time synchronization. We have to calculate guest
> offset and then subtract with host time .
I assume that we can do that by passing the offset of the timestamp
information with guest trace data (that is the simplest implementation).
However, currently trace-merge depends on x86 TSC, but some intel
processors TSC is not stable. That is current barrier of trace-merge.
Thank you,
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hi...@hitachi.com
Divya Vyas
Jan 22, 2015, 5:20:05 AM1/22/15
to
Hi,
I am working on merge of ftrace data of guest and host .
tsc_guest = tsc_host +offset
I got offset from /sys/kernel/debug/tracing/instances/tsc_offset/trace
18446739224709377060
Where as my guest and host data is in below format :
sshd-2332 [000] 4813.363939: net_dev_queue: dev=eth0
skbaddr=0xffff88003b78f6e8 len=134
sshd-2332 [000] 4813.363951: net_dev_xmit: dev=eth0
skbaddr=0xffff88003b78f6e8 len=134 rc=0
sshd-2332 [000] 4813.364010: netif_receive_skb: dev=eth0
skbaddr=0xffff880015cbbc00 len=52
sshd-2332 [000] 4813.364042: net_dev_queue: dev=eth0
skbaddr=0xffff88002c086ce8 len=102
sshd-2332 [000] 4813.364053: net_dev_xmit: dev=eth0
skbaddr=0xffff88002c086ce8 len=102 rc=0
trace-cmd-2496 [000] 4813.364098: netif_receive_skb: dev=eth0
skbaddr=0xffff880015cbbc00 len=52
<idle>-0 [000] 4814.384570: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490a00 len=38
<idle>-0 [000] 4816.384541: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490900 len=38
<idle>-0 [000] 4818.384681: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490900 len=38
How can I add the timestamp and offset .Do I need to add directly the numbers ?
On Tue, Jan 13, 2015 at 11:03 AM, Divya Vyas <edivy...@gmail.com> wrote:
> Hi,
>
> Is there any feature in ftrace like synchronizing the timestamp
>
> http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
>
>
>
> On Wed, Jan 7, 2015 at 2:57 PM, Divya Vyas <edivy...@gmail.com> wrote:
>> Hi Masami,
>>
>> Any updates on the merge feature?
>>
>>
>>
>> On Mon, Dec 22, 2014 at 4:36 PM, Divya Vyas <edivy...@gmail.com> wrote:
>>> Hi Masami,
>>>
>>> Thanks for the patches .
>>>
>>> Are you working on this merging feature also
>>> https://lkml.org/lkml/2013/12/16/688 ? the d option ?
>>>
>>> trace-cmd report -i will not give expected output if the time is not
>>> synchronized . From kernel side we can get TSC offset using
>>> tracepoint when guest boots . I think this tracepoint was added from
>>> 3.11 but from user side I got only the trace-merge script which takes
>>> offset and trace and host data . But the format of trace data this
>>> script accepts is little different .
>>>
>>> My end goal is to to merge the traces in chronological order and see
>>> them in kernelshark .
>>>
>>>
>>> Thanks,
>>> Divya
I am working on merge of ftrace data of guest and host .
tsc_guest = tsc_host +offset
I got offset from /sys/kernel/debug/tracing/instances/tsc_offset/trace
18446739224709377060
Where as my guest and host data is in below format :
sshd-2332 [000] 4813.363939: net_dev_queue: dev=eth0
skbaddr=0xffff88003b78f6e8 len=134
sshd-2332 [000] 4813.363951: net_dev_xmit: dev=eth0
skbaddr=0xffff88003b78f6e8 len=134 rc=0
sshd-2332 [000] 4813.364010: netif_receive_skb: dev=eth0
skbaddr=0xffff880015cbbc00 len=52
sshd-2332 [000] 4813.364042: net_dev_queue: dev=eth0
skbaddr=0xffff88002c086ce8 len=102
sshd-2332 [000] 4813.364053: net_dev_xmit: dev=eth0
skbaddr=0xffff88002c086ce8 len=102 rc=0
trace-cmd-2496 [000] 4813.364098: netif_receive_skb: dev=eth0
skbaddr=0xffff880015cbbc00 len=52
<idle>-0 [000] 4814.384570: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490a00 len=38
<idle>-0 [000] 4816.384541: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490900 len=38
<idle>-0 [000] 4818.384681: netif_receive_skb: dev=eth0
skbaddr=0xffff88003a490900 len=38
How can I add the timestamp and offset .Do I need to add directly the numbers ?
On Tue, Jan 13, 2015 at 11:03 AM, Divya Vyas <edivy...@gmail.com> wrote:
> Hi,
>
> Is there any feature in ftrace like synchronizing the timestamp
>
> http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.linuxtools.lttng.help%2Fdoc%2FTrace-synchronization.html
>
>
>
> On Wed, Jan 7, 2015 at 2:57 PM, Divya Vyas <edivy...@gmail.com> wrote:
>> Hi Masami,
>>
>> Any updates on the merge feature?
>>
>>
>>
>> On Mon, Dec 22, 2014 at 4:36 PM, Divya Vyas <edivy...@gmail.com> wrote:
>>> Hi Masami,
>>>
>>> Thanks for the patches .
>>>
>>> Are you working on this merging feature also
>>> https://lkml.org/lkml/2013/12/16/688 ? the d option ?
>>>
>>> trace-cmd report -i will not give expected output if the time is not
>>> synchronized . From kernel side we can get TSC offset using
>>> tracepoint when guest boots . I think this tracepoint was added from
>>> 3.11 but from user side I got only the trace-merge script which takes
>>> offset and trace and host data . But the format of trace data this
>>> script accepts is little different .
>>>
>>> My end goal is to to merge the traces in chronological order and see
>>> them in kernelshark .
>>>
>>>
>>> Thanks,
>>> Divya
0 new messages