Error clearing the GPU buffer when using gmx+plumed

[joan.c...@gmail.com]

Dear all,

I posted this on another GPU-related conversation I opened a while back, but it got deleted (maybe cause I included a link?), so posting again in a new conversation.

I have posted this in the Gromacs forum list too, as I’m not sure on what end the problem is. I recently found a library that may be useful to (somewhat) easily port certain operations to the GPU . The library is called Bandicoot (link removed, but can be found on sourceforge) and it does work more or less straight out of the box (need to make sure you link it properly which I’m still getting my head round to).

I have managed to offload an expensive matrix operation (using a custom CV) using this library and it does run with plumed driver, but the problem comes when I try to run it on Gromacs. The CV is initialised properly, but before it goes onto the calculator, gromacs crashes with the following error. If anybody has any idea how to fix that (and I do realise that’s untested stuff so no problem if not), I will really appreciate it.

‘’‘’
Program: gmx mdrun, version 2023.3-plumed_2.10.0_dev
Source file: src/gromacs/gpu_utils/devicebuffer.cuh (line 294)
Function: clearDeviceBufferAsync<gmx::BasicVector >(gmx::BasicVector**, size_t, size_t, const DeviceStream&)::<lambda()>

Assertion failed:
Condition: stat == cudaSuccess
Couldn’t clear the device buffer. CUDA error #1 (cudaErrorInvalidValue):
invalid argument.
‘’‘’

Thanks a lot,
Joan

[alx.th...@gmail.com]

Hi Joan!

I am not sure if I have the same stuff as you, but I was always getting this error if at the same time a) something else is wrong and b) -update gpu is turned on. Now I know that using update gpu with any Plumed bias is wrong, but... My message here is that it may not necessarily be the problem it is just a message it prints. If you look at the .log file produced by Gromacs, you may find a different error message that could be the original one. For example, a CV getting outside the metad grid, or wrong CV definition, atom ID not found and so on.

But yes, it may indeed be this very error, and if it the case, then I am sorry that I cannot help you.

Best,

Alex

[joan.c...@gmail.com]

Hi Alex,

Thanks for your suggestions. I have run some tests and the same tpr and plumed files run fine with a version of the code that doesn't call the GPU. I also tried removing the bias (just monitoring the CV), and it still crashes at step 0 with the same error.

Cheers,

Joan