fragment's relation to trace

[Mohammad Ewais]

Hello,

I would like to clarify a couple things. First, what exactly is a fragment? Is it a BB? a Trace? can it be both?

I understand that if I do `dr_register_bb_event` (or its drmgr variants) then I would be able to distinguish whether the code is used for BB or trace creation using the `for_trace` argument. However, when I use `dr_register_delete_event` there is no such argument. I am not even sure if it gets called on trace deletion? or just BB deletion?

A similar question for dr_delete_fragment, does it delete BBs with this tag only? or also traces that include the BB of that tag?

Thanks

[Mohammad Ewais]

Essentially, I am facing the following problem.

During my instrumentation (using drmgr_register_bb_app2app_event) , I keep track of the BBs getting created and stored in the code cache. I understand that the same basic block could exist more than once, so I keep an active count for each BB.

When BBs are deleted (using dr_register_delete_event) I decrement that active count, and if it reaches 0 I just get the BB off my active list.

Occasionally, I have to "reset" my instrumentation, and in an effort to avoid `dr_flush_region` and the like, I instead try to do use `dr_delete_fragment` which should be good because I already know which BBs are currently alive and can specifically delete those. Also because I am running with -thread_private. Something like this:

void* drctx = dr_get_current_drcontext();
for (const auto& [tag, val]: *tls->sim_bb_addresses)
{
    dr_delete_fragment(drctx, tag);
}

 

This occasionally results in segmentation faults from clean calls in supposedly deleted BBs. However, if I run the same code with -disable_traces, I don't get any segfaults. Also, if I switch all the dr_delete_fragment to dr_flush_region, like so:

void* drctx = dr_get_current_drcontext();
dr_mcontext_t mcontext = {sizeof(mcontext), DR_MC_ALL};
dr_get_mcontext(drctx, &mcontext);
mcontext.pc = (byte*)pc;
for (const auto& [tag, val]: *tls->sim_bb_addresses)
{
    app_pc address = dr_fragment_app_pc(tag);
    dr_flush_region(address, 1);
}
dr_redirect_execution(&mcontext);

it also works fine (obviously, not efficient as it would delete the code cache for all threads instead of only the target thread).

So based on this, I am inclined to believe that when I do dr_delete_fragment, I am missing something, maybe traces or something to that effect. Which messes up my instrumentation and causes segfaults.

[Derek Bruening]

On Fri, Nov 10, 2023 at 11:37 AM Mohammad Ewais <mohammad...@gmail.com> wrote:

Hello,

I would like to clarify a couple things. First, what exactly is a fragment? Is it a BB? a Trace? can it be both?

Both.

 

I understand that if I do `dr_register_bb_event` (or its drmgr variants) then I would be able to distinguish whether the code is used for BB or trace creation using the `for_trace` argument. However, when I use `dr_register_delete_event` there is no such argument. I am not even sure if it gets called on trace deletion? or just BB deletion?

Any fragment deletion, bb or trace.

 

A similar question for dr_delete_fragment, does it delete BBs with this tag only? or also traces that include the BB of that tag?

Only the fragment for that thread with that tag (there can be only one).  Does not delete any other fragment that might duplicate that block (tail-duplicate, or trace that includes it not at the start).

 

Thanks

--
You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/45a10a78-6a32-45ee-8b60-de1f660dfb71n%40googlegroups.com.

[Derek Bruening]

Deleting the fragment with that tag won't necessarily remove all copies of that app code.  Yes, flushing is expensive; one alternative for instrumentation with a handful of separate choices is to use drbbdup.  The drmemtrace framework now uses drbbdup to switch between different forms of instrumentation.  We have found this to be superior to flushing.

--

You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/abf01da6-10d4-4746-a081-ec073f3f7731n%40googlegroups.com.

[Mohammad Ewais]

Hi Derek,

Thanks for your help. Just clarifying on the use of drbbdup, do you mean essentially duplicating all BBs during instrumentation, and inserting branches between them to select the appropriate path?

[Derek Bruening]

Yes, drbbdup does that for you.

To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/0fad36f5-100b-4a38-84d5-5973b70d5558n%40googlegroups.com.

[Mohammad Ewais]

Sorry for the so many questions, just making sure I fully understand.

So this is how it works, correct me if I am wrong:

1. BBs get created as they get chosen by the flow of the program, in which case dr_register_bb_event will get called with for_trace = false. These can also get deleted at anytime depending on code cache usage, which I can capture through dr_register_delete_event.
2. As DR finds sequences of BBs that get executed together, it will start creating traces, for which dr_register_bb_event will get called with for_trace = true
3. If I understand correctly, once all the for_trace BBs have been instrumented, they perhaps get copied into the new trace, but their individual BB copies (from #2 above, not from #1) are deleted, which I can capture through dr_register_delete_event.
4. The trace created through #2 and #3 above is constructed and I can capture that through dr_register_trace_event. Maybe this step happens before #3.
5. Traces get modified as needed, which can be captured through dr_register_end_trace_event? Its documentation is a little confusing.

6. When traces are deleted, do I capture that through dr_register_delete_event or dr_register_end_trace_event?

Thanks

[Mohammad Ewais]

I am asking this because I think my problems extend beyond the flushing issue. I think I don't handle trace creation/deletion correctly, and some metadata I keep that depend on the lifetime of BBs are thus sometimes deleted when they shouldn't be.

[Mohammad Ewais]

One more question, I have found BB's tags to usually be the same as the BB app address. I am looking deeper into tags now, and I have noticed that a tag also uses the first BB's app address as its tag!

On a dr_register_delete_event call how do I know if the tag I am receiving is that of BB or a Trace if they are both the same?!

[Derek Bruening]

I don't remember hitting issues with trace vs block identification, but some of our larger tools disable traces for simplicity (e.g., Dr. Memory does that) and other large ones (like drmemtrace) don't really track per-block data.

If a trace is created for tag X, the block at tag X is deleted: there cannot exist two visible fragments at the same sharing level with the same tag.  But, delayed deletion can mean there's a (shadowed) shared block not actually deleted until after the trace has been created.  I know we often use a combination of the delete event plus looking for a new fragment with the same tag, and IIRC the delete event docs recommend using counters.  But it does seem like the block vs trace distinction is there.  If you would like to add an augmented delete event that provides more information, that sounds useful: please file a feature request and send a PR pointing at the feature issue.

To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/39d6da6d-8121-40c2-932b-241e9bb90e94n%40googlegroups.com.