trace emulation of cuda runtime applications

[ash]

Hi everyone,

I am new to Ocelot and I am trying to understand the code while doing
a mini-project in my grad school. I am trying to write a trace
emulator for cuda runtime applications in windows. I stumbled upon
your project while looking for solutions to some arcane issues in cuda
runtime. I am voicing my queries and hope one of you Gurus can help me
out here..!

I am basically doing a simple DLL injection and API interception in
windows c++ to capture the traces of a cuda runtime application in
execution. While playing the application back from the trace file I am
stuck at cudaMalloc as it gives me cuda Error code 11 which I found
out was "error invalid value".

I used the new cubin format specified in ocelot to save the cubin
pointer to cudaRegisterFatBinary and use it in trace emulation .. For
CudaRegisterFunction I am still not sure what the hostFunction,device
function and device name parameters refer to and how I could populate
them. Since this function has no return values I am not sure if it is
invoked fine. Further I am also assuming that these are the only two
entry point functions in a typical runtime applications before I can
make a call to cudaMalloc. Am i right? What am I missing?

please guide me!

[Gregory Diamos]

On 10/14/2011 02:21 PM, ash wrote:
> Hi everyone,
>
> I am new to Ocelot and I am trying to understand the code while doing
> a mini-project in my grad school. I am trying to write a trace
> emulator for cuda runtime applications in windows. I stumbled upon
> your project while looking for solutions to some arcane issues in cuda
> runtime. I am voicing my queries and hope one of you Gurus can help me
> out here..!
>
> I am basically doing a simple DLL injection and API interception in
> windows c++ to capture the traces of a cuda runtime application in
> execution. While playing the application back from the trace file I am
> stuck at cudaMalloc as it gives me cuda Error code 11 which I found
> out was "error invalid value".

You should be checking the error after each API function if you are
not doing this already. Later API calls will sometimes report errors
that occurred previously.

> I used the new cubin format specified in ocelot to save the cubin
> pointer to cudaRegisterFatBinary and use it in trace emulation .. For
> CudaRegisterFunction I am still not sure what the hostFunction,device
> function and device name parameters refer to and how I could populate
> them. Since this function has no return values I am not sure if it is
> invoked fine. Further I am also assuming that these are the only two
> entry point functions in a typical runtime applications before I can
> make a call to cudaMalloc. Am i right? What am I missing?

You can call cudaGetLastError() to see if one of the undocumented API
calls succeeded.

For cudaRegisterFunction, the meaning of the parameters is as follows:

1) hostFun is a key that is used to identify the kernel. This key will
eventually be passed to cudaLaunch. The runtime should lookup
the key to determine which kernel to launch.
2) deviceFun is the (mangled) name of the kernel.
3) I have no idea what the other parameters are, and Ocelot
just ignores them.

Here's how it is implemented in ocelot:

void cudaRegisterFunction(
void **fatCubinHandle,
const char *hostFun,
char *deviceFun,
const char *deviceName,
int thread_limit,
uint3 *tid,
uint3 *bid,
dim3 *bDim,
dim3 *gDim,
int *wSize) {

size_t handle = (size_t)fatCubinHandle;

_lock();

void *symbol = (void *)hostFun;
std::string kernelName = deviceFun;
std::string moduleName = _fatBinaries[handle].name();

report("Registered kernel - " << kernelName
<< " in module '" << moduleName << "'");
_kernels[symbol] = RegisteredKernel(handle, moduleName, kernelName);

_unlock();

> please guide me!
>

[ash]

1) cudaGetLastError returns 0 for both cudaRegisterFatBinary and cudaRegisterFunction

2)Thanks for the information on the parameters of cudaRegisterFunction. From what I could perceive there is no way cudaRegisterFunction's parameters could affect the proper execution of cudaMalloc right?

3) One more question about hostFun parameter. The key that you are talking about should be in a symbol table like thing right? is that part of the fat binary file that is fed to the cudaRegisterFatBinary API ?

Thank you for your reply...!

[ash]

Also the new cubin structure from gpuocelot ..is this compatible with the cubin structure in windows? Because when i tried to parse a pointer to the structure( which i got by intercepting a runtime application exeuction) I couldn't parse the last member variable which was a char pointer .(char *f)

[Gregory Diamos]

On 10/18/2011 10:49 AM, ash wrote:
> 1) cudaGetLastError returns 0 for both cudaRegisterFatBinary and
> cudaRegisterFunction
> 2)Thanks for the information on the parameters of
> cudaRegisterFunction. From what I could perceive there is no way
> cudaRegisterFunction's parameters could affect the proper execution of
> cudaMalloc right?

I would imagine that you could corrupt the runtime by feeding it bad
values. You might want to try not calling it at all to see if it
resolves your issue with cudaMalloc.

>
> 3) One more question about hostFun parameter. The key that you are
> talking about should be in a symbol table like thing right? is that
> part of the fat binary file that is fed to the cudaRegisterFatBinary API ?
>

cudaRegisterFunction is used to create the mapping in the symbol table.
The fat binary will refer to a kernel by its name, say "sgemm".
cudaRegisterFunction is used to create a mapping from a key to
"sgemm". The key will eventually be passed to cudaLaunch.

> Thank you for your reply...!

> --
> You received this message because you are subscribed to the Google
> Groups "gpuocelot" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/gpuocelot/-/JmuoaSEJCEUJ.
> To post to this group, send email to gpuo...@googlegroups.com.
> To unsubscribe from this group, send email to
> gpuocelot+...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/gpuocelot?hl=en.

[Gregory Diamos]

> was a char pointer .(char *f) --

I've not tried this on windows.

> You received this message because you are subscribed to the Google
> Groups "gpuocelot" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/gpuocelot/-/Ila4fc8pE7kJ.