[Question] TRACEPOINTs for modules and apps

[Wonsup Yoon]

Hi,

Is it possible to use TRACEPOINTs for modules or apps?

I've tested, but it does not working for me.

Best regards,

Wonsup Yoon

[Waldek Kozaczuk]

I think it should be possible given this file which is part of java app has tracepoints - 

modules/java-base/balloon/jvm_balloon.cc.

But I have not tested it myself.

Can you give us your example?

Waldek

[Wonsup Yoon]

This is my example code.

#include <stdio.h>

#include <osv/trace.hh>

TRACEPOINT(trace_hello, "i=%d", int);

int main(){

printf("Hello from C code\n");

printf("enabled: %lx\n", trace_hello.enabled());

for(int i =0; i<128;++i){

trace_hello(i);

}

return 0;

}

It seems that TRACEPOINT use tracepoint_patch_sites sections which are linked to loader. (arch/x64/arch-trace.hh)

However, application or modules are dynamically linked, so their tracepoint are not visible to GDB.

loader.ld's tracepoint_patch_sites:

  .tracepoint_patch_sites ALIGN(8) : AT(ADDR(.tracepoint_patch_sites) - OSV_KERNEL_VM_SHIFT) {

        __tracepoint_patch_sites_start = .;

        *(.tracepoint_patch_sites)

        __tracepoint_patch_sites_end = .;

    } : text

2020년 9월 8일 화요일 오후 1시 49분 35초 UTC+9에 jwkoz...@gmail.com님이 작성:

[Nadav Har'El]

I have to admit it's been a long while (a few years) since I looked deeply into our tracepoint implementation, so I'm quite rusty.

I thought that in the past OSv tracepoints *did* work in modules - a good example is the test tests/tst-tracepoint - but I'm not sure this test actually tests everything you want to work (such as dynamically enabling or disabling specific tracepoints?) - can you please clarify what did you try and didn't work?

On Mon, Sep 21, 2020 at 1:32 PM Wonsup Yoon <pus...@kaist.ac.kr> wrote:

This is my example code.

#include <stdio.h>

#include <osv/trace.hh>

TRACEPOINT(trace_hello, "i=%d", int);

int main(){

printf("Hello from C code\n");

printf("enabled: %lx\n", trace_hello.enabled());

for(int i =0; i<128;++i){

trace_hello(i);

}

return 0;

}

It seems that TRACEPOINT use tracepoint_patch_sites sections which are linked to loader. (arch/x64/arch-trace.hh)

However, application or modules are dynamically linked, so their tracepoint are not visible to GDB.

How is GDB relevant here?

If you haven't seen this already, check out https://github.com/cloudius-systems/osv/wiki/Debugging-OSv#tracepoints explaining how to use tracepoints.

When an executable is loaded (either the main OSv kernel or a loadable object), the constructors of all the global objects

it has are run. The tracepoint's constructor, tracepoint_base::tracepoint_base(), adds itself to a global list of trace points

(tp_list) and also considers whether to enable this tracepoint. The reverse happens when the object is unloaded.

What kind of problems do you experience when you try to enable or use such tracepoints?

I wonder if the problem is that the tracepoints *code* (so-called "text segment") is marked read-only so we can't modify it. I don't remember what

we did about this in the past. If this is the problem you should expect segfaults when trying to enable the tracepoints. Is this what you are seeing?

loader.ld's tracepoint_patch_sites:

  .tracepoint_patch_sites ALIGN(8) : AT(ADDR(.tracepoint_patch_sites) - OSV_KERNEL_VM_SHIFT) {

        __tracepoint_patch_sites_start = .;

        *(.tracepoint_patch_sites)

        __tracepoint_patch_sites_end = .;

    } : text

loader.ld only applies to the main Scylla executable, not to shared objects.

The only thing this loader.ld thing does is to create the __tracepoint_patch_sites_start, __tracepoint_patch_sites_end variables. I don't remember why we actually need those...

[Wonsup Yoon]

My problem is unable to list and view TRACEPOINTs in application part using trace.py.

Command: ./scripts/run.py --trace=hello

Get Trace: scripts/trace.py extract && scripts/trace.py list (nothing shows up, it uses GDB to extract tracepoints)

I think it is due to there are no tracepoint_patch_site entries for TRACEPOINTs in application codes, and they are not correctly activated during initialization (trace.cc:265) . 

The activation seems to be related to fast path of TRACEPOINTs.

Removing fast path part of x64/arch-trace.hh solves this problem.

tst-tracepoint only tests serialization of signatures and arguments.

This does not guarantee correct operation.

2020년 9월 21일 월요일 오후 9시 49분 21초 UTC+9에 Nadav Har'El님이 작성: