Running multiple caffemodels for predictions on multiple images on one GPU in parallel

[Abhinav Kumar Gupta]

Hi,

We are able to run one caffe model on multiple images on one GPU. 

We were not able to run multiple caffe models on multiple images on one GPU for making predictions in parallel. We wanted to check if that is possible.

The models are pretty light and we are using NVIDIA 980

Thank,
Abhinav

[Jan C Peters]

As long as you are running caffe (or pycaffe) in different OS processes that shouldn't be a problem (I am running multiple caffe instances for training on a single GPU right now). The only points you may get in trouble are:
- running short of available memory on the GPU if you have too many processes/too large batchsizes or sample dimensions
- if you are using the same sample db for input for multiple caffe instances, you probably can't use LevelDB (afaik it has no read-only mode and only allows exclusive access). But for LMDB and H5 that shouldn't be a problem.