Check failed while creating lmdb/leveldb datasets : mdb_status == 0 (87 vs. 0)

[henok g]

Creating train lmdb...
*** Check failure stack trace: ***
I0611 18:19:32.873293  5688 convert_imageset.cpp:83] Shuffling data
I0611 18:19:33.263316  5688 convert_imageset.cpp:86] A total of 12961 images.
I0611 18:19:33.269316  5688 db_lmdb.cpp:39] Opened lmdb C:/svn/viNet/caffe-windows/examples/retina//retina-train-lmdb-3class
F0611 18:31:43.033056  5688 db_lmdb.hpp:14] Check failed: mdb_status == 0 (87 vs. 0) The parameter is incorrect.
Creating val lmdb...
*** Check failure stack trace: ***
I0611 18:32:38.282217  6088 convert_imageset.cpp:83] Shuffling data
I0611 18:32:45.277616  6088 convert_imageset.cpp:86] A total of 7070 images.
I0611 18:32:45.371623  6088 db_lmdb.cpp:39] Opened lmdb C:/svn/viNet/caffe-windows/examples/retina//retina-val-lmdb-3class
F0611 18:48:13.779724  6088 db_lmdb.hpp:14] Check failed: mdb_status == 0 (87 vs. 0) The parameter is incorrect.
Done. Press <enter> to exit ...

The error seams to point to the amount of memory I have on my system ( 12 GB ) but i am not sure. If I take few images the commands run just fine but if I use the entire data set, 12961 training images and 7070 test images, the operation would halt with the message above. Any suggestions? Thanks for taking the time to read / & reply to this post.

[navee k]

Hey..

Did you get a chance to root cause this issue????

Thanks,

Navee

[Mattia Lupetti]

Hi, I also have this problem with LMDB. The problem does not depend on the size of data to be written, nor on the frequency of Commits and NewTransactions. It looks like when the database size reaches about 2 Gb, the writing crashes. Any idea?

Thanks,

Mattia

[Albert Diao]

I also have the same problem, when the mdb file is growing to 2.02 GB

在 2016年10月25日星期二 UTC+8下午3:30:38，Mattia Lupetti写道：

[Albert Diao]

I have solved this problem, it takes me a whole day...

error 87 is CURSSOR_FULL, It may be an bug of old version lmdb, when file size up to 2GB.

I download and rebuild a  new lmdb (from https://github.com/willyd/caffe-windows-dependencies), It works.

By the way, different from the old one, new lmdb malloc disk space at first, due to the LMDB_MAP_SIZE you set, even your dataset's real size is smaller.

在 2016年11月2日星期三 UTC+8下午1:09:24，Albert Diao写道：

[Daniel Moodie]

I'm fairly certain this is a bug with lmdb as I've not been able to build a database over 4GB on windows.  I can build one on Linux without a problem though.

[Guillaume Dumont]

Can you try the version of lmdb included in the caffe-builder project and tell me if it works?

[ypaafw]

I tried this method, add the file(from https://github.com/willyd/caffe-windows-dependencies) to the 3rdparty, but failed. Could you tell me these special steps?Thank you
在 2016年11月2日星期三 UTC+8下午11:16:17，Albert Diao写道：

[lunzueta]

Hi Guillaume. I can confirm that it doesn't work with the latest version of lmdb included in caffe-builder, but this works in Linux (Ubuntu 16.04). It would be great if this could also work in Windows, but unfortunately, I can't find where the problem is.

[Guillaume Dumont]

I had the same problem and figured out that the problem was due to long being 64-bit on Linux and 32-bit on Windows. I have patched the caffe-builder LMDB and vcpkg has a port for LMDB that includes the patch. With either of these it should be possible to create a DB larger than 2GB.