Check failed: error == cudaSuccess (2 vs. 0) out of memory despite not exceeding GPU memory?

[Kwot Sin Lee]

Hi all, I'm getting the error: "Check failed: error == cudaSuccess (2 vs. 0)  out of memory", although my GPU should have sufficient memory to run without reducing the batch size or dimensions of the data. I checked using nvidia-smi, and caffe was only using slightly over 1 GB of GPU memory. My total GPU memory is 4GB, and all other secondary irrelevant processes in my system were taking only a maximum of 200MB of GPU memory. Why is my GPU memory still insufficient?