One lmdb two experiment

[Yusuf Tas]

Hi everyone,

    I have 2 GPUs to run 2 test at the same time  for parameter optimization.  Do I need to create two separate LMDB files for two GPU, or can they share the same one ? Would it cause any race conditions, problems ? My guess is no, since they are 2 completely different processes. But if two process request to read at the same time, there might be some small waiting occur, I guess ?


Regards,
Yusuf

[Jan]

As long as they only read from the lmdb (which is what the data layers in caffe do), that shouldn't be a problem. However this does not work with LevelDB, as that kind of DB can only be accessed by process at a time. So yeah, crweate one LMDB and let the processes share it. As to the performance I cannot really comment, you should try it, but I don't think this really affects performance very much: In reality it is probably only read once from disk and then only read from the cache by the second process.

Jan

[Yusuf Tas]

Thanks Jan for your answer, I appreciate it. I didn't know it was different for leveldb.

Yusuf