Tracing programs which are not CLI based

100 views
Skip to first unread message

ByteCrew

unread,
Sep 26, 2022, 9:55:09 AM9/26/22
to DynamoRIO Users
Hi, I'm actually using DRIO to monitor and log process behaviour (in term of system calls issued). My goal is to retrieve a syscall log with my client WHILE the monitored process is running. For instance I tried to attach my client by doing 

ddrun.exe -attach <PID> -c <myclient.dll> 

but it keeps crashing, sometimes it gives me an error msg which states only "program aborted". I know that's an experimental features, etc 
I was wondering if there is a way to start a process with DRIO without having it closed immediately after the command is done. I'm able to monitor CLI applications because before I start drrun.exe I'm able to pass some parameters (for example I'm able to monitor cmd.exe because I can pass arguments to it, but what about chrome browser? How can I monitor chrome without passing any argument via CLI?)

If I go

drrun.exe -c <myclient.dll> -- <path\to\chrome.exe> 

the command simply returns, chrome opens and closes immediately.

Thank you.

Derek Bruening

unread,
Sep 26, 2022, 11:55:03 AM9/26/22
to ByteCrew, DynamoRIO Users
On Mon, Sep 26, 2022 at 9:55 AM ByteCrew <strozz...@gmail.com> wrote:
Hi, I'm actually using DRIO to monitor and log process behaviour (in term of system calls issued). My goal is to retrieve a syscall log with my client WHILE the monitored process is running. For instance I tried to attach my client by doing 

ddrun.exe -attach <PID> -c <myclient.dll> 

but it keeps crashing, sometimes it gives me an error msg which states only "program aborted". I know that's an experimental features, etc 

I would suggest helping to diagnose the problem(s): what are the Windows and DR versions?  Does attach work with no client on a tiny application?  What is the smallest application and client combination where it fails?  Do the -takeover_sleep or -takeovers parameters help?  Are there useful messages in debug build?  What is the precise crash message?  It may be worth putting this into the Windows attach issue https://github.com/DynamoRIO/dynamorio/issues/725.
 
I was wondering if there is a way to start a process with DRIO without having it closed immediately after the command is done. I'm able to monitor CLI applications because before I start drrun.exe I'm able to pass some parameters (for example I'm able to monitor cmd.exe because I can pass arguments to it, but what about chrome browser? How can I monitor chrome without passing any argument via CLI?)

If I go

drrun.exe -c <myclient.dll> -- <path\to\chrome.exe> 

the command simply returns, chrome opens and closes immediately.

Do you already have chrome running somewhere?  A new launch will notice the already-running instance and will exit, which is probably what is happening here.  To start a new instance you need to point at a different profile; something like --user-data-dir.  This is all chrome-specific: please look up chrome docs.  It is unrelated to DR.
 

Thank you.

--
You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dynamorio-users/d64deb27-0a9a-4d4b-bea3-8dc8f27fdb12n%40googlegroups.com.

ByteCrew

unread,
Sep 27, 2022, 8:38:15 AM9/27/22
to DynamoRIO Users
I would suggest helping to diagnose the problem(s): what are the Windows and DR versions?  Does attach work with no client on a tiny application?  What is the smallest application and client combination where it fails?  Do the -takeover_sleep or -takeovers parameters help?  Are there useful messages in debug build?  What is the precise crash message?  It may be worth putting this into the Windows attach issue https://github.com/DynamoRIO/dynamorio/issues/725.

Win10 19044, DRIO latest version. Yes, it does attach on applications without clients. The smallest client combination is my client (which uses drsyscall + other syscall related routines) attached on notepad. I'm going to try the takeover options.
 
Do you already have chrome running somewhere?  A new launch will notice the already-running instance and will exit, which is probably what is happening here.  To start a new instance you need to point at a different profile; something like --user-data-dir.  This is all chrome-specific: please look up chrome docs.  It is unrelated to DR.

 Actually I'm 100% sure that the instance of chrome running is the only one currently executing.

Does the fact that the -attach option working fine without a client means something?
Message has been deleted
Message has been deleted
Message has been deleted

ByteCrew

unread,
Sep 28, 2022, 10:45:09 AM9/28/22
to DynamoRIO Users
With respect to 

Do you already have chrome running somewhere?  A new launch will notice the already-running instance and will exit, which is probably what is happening here.  To start a new instance you need to point at a different profile; something like --user-data-dir.  This is all chrome-specific: please look up chrome docs.  It is unrelated to DR.

maybe I'm getting more into this: if I spawn a process with 

drrun.exe -c <client.dll> -- notepad.exe  

WITH, in my main:

drsys_options_t ops = { sizeof(ops), 0, };
drsys_init(id, &ops);

AND in my exit exit event: drsys_exit( ) 

Everything goes fine, which means that the instance of - in this case - notepad.exe is launched but stills closes immediately.
Before this test I didn't have drsys_exit( ) call in my exit event and I was getting memory leaks errors. 
Now the error is fixed but I still don't understand why the instance can't just remain opened. Obviously if I remove all the drsys initialization it works. Any hints? 

sharma...@google.com

unread,
Sep 29, 2022, 10:29:40 AM9/29/22
to DynamoRIO Users
> Everything goes fine, which means that the instance of - in this case - notepad.exe is launched but stills closes immediately.

Are you sure there was no error?

Did you try the debug build with logs (https://dynamorio.org/page_logging.html)? Does it output anything useful? Does it hit any assertion?

Abhinav

Message has been deleted

ByteCrew

unread,
Sep 29, 2022, 10:44:04 AM9/29/22
to DynamoRIO Users
Are you sure there was no error?

If I open processExplorer the only thing I see is that drrun launches the instance, but after less than a second they both close.
Like I said before if I remove all the drsys' stuff the instrumentation works, which means that the launched app remains alive. 

The command drrun.exe -debug -c <myclient.dll> -- mspaint.exe returns:

<CURIOSITY : instr_get_opcode(instr_new) != instr_get_opcode(instr_old) in file D:\a\dynamorio\dynamorio\core\win32\callback.c line 2080
version 9.0.1, custom build
-no_dynamic_options -client_lib 'C:\Users\...\Desktop\systracer.dll;0;' -client_lib64 'C:\Users\...\Desktop\systracer.dll;0;' -code_api -probe_api -stack_size 56K -max_elide_jmp 0 -max_elide_call 0 -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct
C:\Users\...\Desktop\DynamoRIO\lib64\debug\dynamorio.dll=0x0000000015000000
C:\Users\...\Desktop\systracer.dll=0x00007ff6b13f0000
C:\Users\...\Desktop\DynamoRIO\drmemory/drmf\lib64\debug/drsyscall.dll=0x00007ff6b1410000
C:\Users\...\Desktop\DynamoRIO\ext\lib64\debug/drsyms.dll=0x00007ff6b16e0000
C:\Users\...\Desktop\DynamoRIO\ext\lib64\debug/dbghelp.dll=0x00007ff6b1770000
C:\Windows/system32/ucrtbase.dll=0x0000020e9fea0000
C:\Windows/system>
<CURIOSITY : instr_new == instrlist_first(ilist) || instr_new == instr_get_next(instrlist_first(ilist)) in file D:\a\dynamorio\dynamorio\core\win32\callback.c line 2083
version 9.0.1, custom build
-no_dynamic_options -client_lib 'C:\Users\...\Desktop\systracer.dll;0;' -client_lib64 'C:\Users\...\Desktop\systracer.dll;0;' -code_api -probe_api -stack_size 56K -max_elide_jmp 0 -max_elide_call 0 -no_inline_ignored_syscalls -native_exec_default_list '' -no_native_exec_managed_code -no_indcall2direct
C:\Users\...\Desktop\DynamoRIO\lib64\debug\dynamorio.dll=0x0000000015000000
C:\Users\...\Desktop\systracer.dll=0x00007ff6b13f0000
C:\Users\...\Desktop\DynamoRIO\drmemory/drmf\lib64\debug/drsyscall.dll=0x00007ff6b1410000
C:\Users\...\Desktop\DynamoRIO\ext\lib64\debug/drsyms.dll=0x00007ff6b16e0000
C:\Users\...\Desktop\DynamoRIO\ext\lib64\debug/dbghelp.dll=0x00007ff6b1770000
C:\Windows/system32/ucrtbase.dll=0x0000020e9fea0000
C:\Windows/system>
<Cleaning hooked Nt wrapper @0x00007ff973210960 sysnum=0x1c0>
<curiosity: rex.w on OPSZ_6_irex10_short4!>
<Stopping application C:\Windows\system32\mspaint.exe (11856)>
<Failed to suspend attached-but-never-scheduled thread 5584>
<Failed to suspend attached-but-never-scheduled thread 11756>
<Failed to suspend attached-but-never-scheduled thread 20992>

I have only one user on the windows machine that I'm using, OS major is 19044 and I'm sure that no other instance is runnin when I issue the command. 

How can I check if there could be errors that I'm not currently noticing?

Thank you. 

sharma...@google.com

unread,
Sep 29, 2022, 11:54:45 AM9/29/22
to DynamoRIO Users
The  "Failed to suspend attached-but-never-scheduled" logs are curious but they might not be important because they occur after the "Stopping application" log line (which is printed when the app is about to exit: https://github.com/DynamoRIO/dynamorio/blob/144a9d9222a1fbc8d1c4ffe75b5eff8dfcb8ad7f/core/dynamo.c#L1432).

Could you check the logfiles (https://dynamorio.org/page_logging.html)? Also make sure that op_verbose_level (https://github.com/DynamoRIO/drmemory/blob/3ad8c8856254fa86d5bb2161678e23533b74cf5b/drmemory/frontend.c#L999) is set to highest (using "-vvv"), which I think decides the verbosity of the LOGs in drsyscall.

Could you also try running a different application where we can more easily identify an abnormal exit? E.g. some app that is supposed to produce some visible command-line output.

Abhinav

ByteCrew

unread,
Sep 29, 2022, 1:40:51 PM9/29/22
to DynamoRIO Users
Could you check the logfiles (https://dynamorio.org/page_logging.html)? Also make sure that op_verbose_level (https://github.com/DynamoRIO/drmemory/blob/3ad8c8856254fa86d5bb2161678e23533b74cf5b/drmemory/frontend.c#L999) is set to highest (using "-vvv"), which I think decides the verbosity of the LOGs in drsyscall.

 Log files generated with a standard -loglevel 2 are huge and are not easy to share (there are 4 log files after one execution). What am I looking for in those files? Maybe I can share the entire .log file here.
Plus, I did not understand how the -vvv parameter should be used, it's not a drrun parameter I guess since it gave me errors. 

It may be worth specifying that my client is "written" with dynamorio and not with drmemory. I import dr_syscall.h to use it in my client, I don't know if it's a meaningful info.
Here's my cmake:

add_library(systracer SHARED systracer.c)
find_package(DynamoRIO)
find_package(DrMemoryFramework)
if (NOT DynamoRIO_FOUND)
message(FATAL_ERROR "DynamoRIO package
required to build")
endif(NOT DynamoRIO_FOUND)
configure_DynamoRIO_client(systracer)
use_DynamoRIO_extension(systracer drsyscall)
use_DynamoRIO_extension(systracer drmgr)
use_DynamoRIO_extension(systracer drwrap)
#use_DynamoRIO_extension(dbitool drreg)
#use_DynamoRIO_extension(dbitool drx)
#use_DynamoRIO_extension(dbitool drutil)
#use_DynamoRIO_extension(dbitool drwrap)

 
Could you also try running a different application where we can more easily identify an abnormal exit? E.g. some app that is supposed to produce some visible command-line output.

Like I said in the first post it works only with CLI applications because before issuing ddrun.exe [...] I specify the arguments before. I tried notepad, calc, mspaint, brave browser and procexp but all of them have the exact same behaviour. No error, no crash msgbox whatsoever. Nothing.

Thank you.
 
notepad.exe_log.html

Derek Bruening

unread,
Sep 29, 2022, 1:50:34 PM9/29/22
to ByteCrew, DynamoRIO Users
I would suggest:

1) Try something else that uses drsyscall, like drstrace. Does it work on notepad on your platform.
2) What is the exit code: is it indicating any error.
3) Use either the logfiles or a debugger to observe how the application is exiting and try to see what is triggering a regular but premature exit.
4) If somehow the drsyscall events are causing a premature exit, try disabling each piece of dryscall to zero in on what is causing this.

--
You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.

ByteCrew

unread,
Sep 29, 2022, 2:23:59 PM9/29/22
to DynamoRIO Users
1) Try something else that uses drsyscall, like drstrace. Does it work on notepad on your platform.
 
drstrace works

2) What is the exit code: is it indicating any error.

DRIO's exit code? Where can I check this? 

3) Use either the logfiles or a debugger to observe how the application is exiting and try to see what is triggering a regular but premature exit.
 
What should I look for inside the logs? Like I said before the logs generated by a single execution are several and a couple of them are huge. I did not notice anything useful. 

4) If somehow the drsyscall events are causing a premature exit, try disabling each piece of dryscall to zero in on what is causing this.

I use drsyscall ONLY to retrieve syscall numbers and names, If I could do this without using dr_syscall I would replace it in zero-time because this bug is giving me headaches since thursday :((
If I comment out every piece of drsyscall-related code everything works fine.
 
It may be worth specifying that my client is "written" with dynamorio and not with drmemory. I import dr_syscall.h to use it in my client, I don't know if it's a meaningful info.
Here's my cmake:

add_library(systracer SHARED systracer.c)
find_package(DynamoRIO)
find_package(DrMemoryFramework)
if (NOT DynamoRIO_FOUND)
message(FATAL_ERROR "DynamoRIO package
required to build")
endif(NOT DynamoRIO_FOUND)
configure_DynamoRIO_client(systracer)
use_DynamoRIO_extension(systracer drsyscall)
use_DynamoRIO_extension(systracer drmgr)
use_DynamoRIO_extension(systracer drwrap)
#use_DynamoRIO_extension(dbitool drreg)
#use_DynamoRIO_extension(dbitool drx)
#use_DynamoRIO_extension(dbitool drutil)
#use_DynamoRIO_extension(dbitool drwrap)

Is everything fine with what has been said here?

Thank you. 
 

Derek Bruening

unread,
Sep 30, 2022, 10:40:07 AM9/30/22
to ByteCrew, DynamoRIO Users
On Thu, Sep 29, 2022 at 2:24 PM ByteCrew <strozz...@gmail.com> wrote:

1) Try something else that uses drsyscall, like drstrace. Does it work on notepad on your platform.
 
drstrace works

If drstrace works with drsyscall but your client does not, you now have two points and can search between them to find the reason for failure.
 

2) What is the exit code: is it indicating any error.

DRIO's exit code? Where can I check this? 

The app's exit code.  Check it how you would normally check an app exit code: depends on your shell.  The exit code is the first thing to check on an early exit to see if the app thinks it's an erroneous or normal exit.
 

3) Use either the logfiles or a debugger to observe how the application is exiting and try to see what is triggering a regular but premature exit.
 
What should I look for inside the logs? Like I said before the logs generated by a single execution are several and a couple of them are huge. I did not notice anything useful. 

What code led to the process exiting early is what you're looking for.  It might be easier in a debugger if the log symbols (probably just nearest exported symbol) are not meaningful enough or the syscall sequence in the log doesn't tell you enough.  At least check whether the exit is a regular NtTerminateProcess or some kind of fault that didn't show up visibly for some reason.
 

4) If somehow the drsyscall events are causing a premature exit, try disabling each piece of dryscall to zero in on what is causing this.

I use drsyscall ONLY to retrieve syscall numbers and names, If I could do this without using dr_syscall I would replace it in zero-time because this bug is giving me headaches since thursday :((
If I comment out every piece of drsyscall-related code everything works fine.
 
It may be worth specifying that my client is "written" with dynamorio and not with drmemory. I import dr_syscall.h to use it in my client, I don't know if it's a meaningful info.
Here's my cmake:

add_library(systracer SHARED systracer.c)
find_package(DynamoRIO)
find_package(DrMemoryFramework)
if (NOT DynamoRIO_FOUND)
message(FATAL_ERROR "DynamoRIO package
required to build")
endif(NOT DynamoRIO_FOUND)
configure_DynamoRIO_client(systracer)
use_DynamoRIO_extension(systracer drsyscall)
use_DynamoRIO_extension(systracer drmgr)
use_DynamoRIO_extension(systracer drwrap)
#use_DynamoRIO_extension(dbitool drreg)
#use_DynamoRIO_extension(dbitool drx)
#use_DynamoRIO_extension(dbitool drutil)
#use_DynamoRIO_extension(dbitool drwrap)

Is everything fine with what has been said here?

Thank you. 
 

--
You received this message because you are subscribed to the Google Groups "DynamoRIO Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dynamorio-use...@googlegroups.com.

ByteCrew

unread,
Oct 3, 2022, 4:39:35 AM10/3/22
to DynamoRIO Users

If drstrace works with drsyscall but your client does not, you now have two points and can search between them to find the reason for failure.

The only difference between drstrace and my client is that I don't use dr_init( ) to initialize drsyscall components but the main method. Beside that the only lines of code that I have are drsys_init, drsys_exit and two drsys_number_to_syscall calls. I really don't understand what is wrong with those 4 lines.

I'm actually replacing dr_syscall with something else since the exit code is 0 and I have nothing left to test apparently. This debug is going way too deeply for a simple number to name conversion. 

Thank you all!!
Reply all
Reply to author
Forward
0 new messages