How to build a forkserver for DynamoRIO's client?

[Yunjie Deng]

I see QEMU can insert the forkserver code into target program so that the forkserver is executed in target program's context (i.e., if fork() calls, the target program fork itself). 

I want to implement the similar thing in DynamoRIO. I find an API 

"void 

dr_insert_clean_call(void *drcontext, instrlist_t *ilist, instr_t *where, void *callee,

                     bool save_fpstate, uint num_args, ...);

" and read the example usage of this API. 

static void

code_cache_init(void)

{

    void *drcontext;

    instrlist_t *ilist;

    instr_t *where;

    byte *end;

    drcontext = dr_get_current_drcontext();

    code_cache =

        dr_nonheap_alloc(page_size, DR_MEMPROT_READ | DR_MEMPROT_WRITE | DR_MEMPROT_EXEC);

    ilist = instrlist_create(drcontext);

    /* The lean procecure simply performs a clean call, and then jumps back. */

    /* Jump back to DR's code cache. */

    where = INSTR_CREATE_jmp_ind(drcontext, opnd_create_reg(DR_REG_XCX));

    instrlist_meta_append(ilist, where);

    /* Clean call */

    dr_insert_clean_call(drcontext, ilist, where, (void *)clean_call, false, 0);

    /* Encode the instructions into memory and clean up. */

    end = instrlist_encode(drcontext, ilist, code_cache, false);

    DR_ASSERT((size_t)(end - code_cache) < page_size);

    instrlist_clear_and_destroy(drcontext, ilist);

    /* Set the memory as just +rx now. */

    dr_memory_protect(code_cache, page_size, DR_MEMPROT_READ | DR_MEMPROT_EXEC);

}

My understand is that dr_insert_clean_call will insert ilist (in my case, forkserver instruction) before where (in my case, target program). And callee will be called fater dr_insert_clean_call finished. I am not sure whether my understand is right nor whether I can build the forkserver through this way.

[assad.hashm...@gmail.com]

dr_insert_clean_call() inserts a call to a C function in the basic-block referred to by ilist. In your snippet above, clean_call is the C function which is called.

Typically dr_insert_clean_call() is called from within a callback function which is registered with an event handler like drmgr_register_bb_instrumentation_event().

DynamoRIO will pass parameters like ilist to the callback function.

You simply have to implement the clean_call C function to do what you want, in your case a forking.

You can insert instructions into ilist before dr_insert_clean_call() calls clean_call but you don't have to if clean_call does everything you want.

Have a look at one of the sample clients which uses clean calls, e.g. api/samples/instrace_simple.c:

static void

clean_call(void)                                        <-- The function called by dr_insert_clean_call()
{
    void *drcontext = dr_get_current_drcontext();
    instrace(drcontext);
}

static dr_emit_flags_t
event_app_instruction(void *drcontext, void *tag, instrlist_t *bb, instr_t *instr,        <-- The callback function
                      bool for_trace, bool translating, void *user_data)
{
 . . .
    /* insert code to add an entry to the buffer */
    instrument_instr(drcontext, bb, instr);

    /* insert code once per bb to call clean_call for processing the buffer */
    if (drmgr_is_first_instr(drcontext, instr)
 . . .
        dr_insert_clean_call(drcontext, bb, instr, (void *)clean_call, false, 0);  <-- Insert call to clean_call at translation time

    return DR_EMIT_DEFAULT;
}

DR_EXPORT void
dr_client_main(client_id_t id, int argc, const char *argv[])
{
. . .
    /* register events */
    dr_register_exit_event(event_exit);
    if (!drmgr_register_thread_init_event(event_thread_init) ||
        !drmgr_register_thread_exit_event(event_thread_exit) ||
        !drmgr_register_bb_instrumentation_event(NULL /*analysis_func*/,   <-- Registering the callback function event_app_instruction
                                                 event_app_instruction, NULL))
        DR_ASSERT(false);
 . . .
}

[Yunjie Deng]

Thank you for your valuable answer. 

I still have a question about the context of clean_call(). It seems that clean_call() will be executed under the DynamoRIO's context rather than the client's context. This means, if a fork is called in clean_call(), my DynamoRIO tools will be forked. However, I only want my client to be forked. 

I am not sure whether the instruction in ilist will be executed under the client's context. If so, I should insert forkserver instructions into ilist.

[assad.hashm...@gmail.com]

An important requirement for DynamoRIO is transparency https://dynamorio.org/transparency.html

In essence this means that when executing a subject program under control of DynamoRIO and a client, the program should behave exactly as it would if it was executed without DynamoRIO. This means that if you call fork() in a client it should fork() the subject program, not DynamoRIO and client.

There are callback functions you can register to do something when forking, see dr_register_fork_init_event() and dr_unregister_fork_init_event()

> However, I only want my client to be forked. 

Why do you want to fork a client?!?

> It seems that clean_call() will be executed under the DynamoRIO's context rather than the client's context.

> I am not sure whether the instruction in ilist will be executed under the client's context.

I think there's some confusion here, in language and/or understanding.

drcontext is an opaque data structure used by DynamoRIO, not something you need to worry about.

Instrumentation code is inserted into the subject program, instrumentation like a clean call and/or a sequence of instructions in ilist.

You should try and do as much instrumentation in a clean call rather than an ilist.

[Assad Hashmi]

> This means that if you call fork() in a client it should fork() the subject program, not DynamoRIO and client

Correction, the DynamoRIO process running the subject program will be forked but due to transparency it will look like the subject program has forked.

But the forked subject program will still be under the control of DynamoRIO.

Derek Bruening or Abhinav Sharma can correct me if I'm wrong.

--
You received this message because you are subscribed to a topic in the Google Groups "DynamoRIO Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dynamorio-users/Ooo2sbqlQkA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dynamorio-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/dc565340-df64-4fa7-81a3-f1f1c8661962n%40googlegroups.com.

[Derek Bruening]

Re: forkserver: a similar concept is implemented in Dr. Fuzz https://drmemory.org/page_drfuzz.html where a target function is executed over and over with different parameter values each time.  IIRC it restores the register values on each iteration, but not memory, a tradeoff which works well for many target functions.

AFL implemented a forkserver using DR.

You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/CALjg6-puc2RoRjszsgwU%2BSxvuxknZJw0uX7bAFH7_Bhg-3dESQ%40mail.gmail.com.