cifar10_train_leveldb

[Zhihong Zeng]

Hello,

I followed the instruction in CIFAR-10 train and test: 

1)  get_cifar10.sh: success

2) create_cifar10.sh: no complain and generated example/cifar10/cifar10-leveldb directory and the data set image mean.binaryproto. In the cifar10-leveldb there are three empty files (lock, log, and log.old).

3) when I run train_quick.sh, and got the follow error:

I1211 15:30:27.533612 22827 net.cpp:56] Memory required for data: 0 I1211 15:30:27.533678 22827 layer_factory.hpp:78] Creating layer cifar I1211 15:30:27.533712 22827 data_transformer.cpp:24] Loading mean file fromexamples/cifar10/mean.binaryproto I1211 15:30:27.534044 22827 net.cpp:67] Creating Layer cifar I1211 15:30:27.534078 22827 net.cpp:356] cifar -> data I1211 15:30:27.534127 22827 net.cpp:356] cifar -> label I1211 15:30:27.534167 22827 net.cpp:96] Setting up cifar I1211 15:30:27.534198 22827 data_layer.cpp:32] Opening dataset examples/cifar10/cifar10_train_leveldb I1211 15:30:27.534219 22827 leveldb_dataset.cpp:13] LevelDB: Open examples/cifar10/cifar10_train_leveldb I1211 15:30:27.534257 22827 leveldb_dataset.cpp:30] mode RO I1211 15:30:27.534292 22827 leveldb_dataset.cpp:43] Opening leveldb examples/cifar10/cifar10_train_leveldb E1211 15:30:27.534489 22827 leveldb_dataset.cpp:49] Failed to open leveldb examples/cifar10/cifar10_train_leveldb. Is it already existing? F1211 15:30:27.534754 22827 data_layer.cpp:33] Check failed: dataset_->open(source, Dataset<string, Datum>::ReadOnly) *** Check failure stack trace: *** @ 0xb714c1ec (unknown) @ 0xb714c11b (unknown)

I think that there is something wrong at step 2), do you have any suggestion?

Thanks in advance,

Zhihong

[Zhihong Zeng]

I found the reasons of the problems:

1) create_cifar10.sh at 

https://github.com/BVLC/caffe/blob/master/examples/cifar10/create_cifar10.sh

"

./build/examples/cifar10/convert_cifar_data.bin $DATA $EXAMPLE

./build/tools/compute_image_mean $EXAMPLE/cifar10_train_leveldb $EXAMPLE/mean.binaryproto leveldb

"

2) create_cifar10.sh at git dev branch which I am using and downloaded a few days ago: 

"

EXAMPLE=examples/cifar10

DATA=data/cifar10

DBTYPE=lmdb

BUILDDIR=./build/caffe

echo "Creating $DBTYPE..."

(***)rm -rf $EXAMPLE/cifar10_train_$DBTYPE $EXAMPLE/cifar10_test_$DBTYPE

$BUILDDIR/examples/cifar10/convert_cifar_data $DATA $EXAMPLE $DBTYPE

echo "Computing image mean..."

(***)$BUILDDIR/tools/compute_image_mean -backend=$DBTYPE \

  $EXAMPLE/cifar10_train_$DBTYPE $EXAMPLE/mean.binaryproto

echo "Done."

"

3) there are two differences between these two files which I highlights by ***. 

So one problem is: convert_cifar_data generate lmdb files in 2) (fail during train ) but generate leveldb files in 1) (success in train).

the second problem is: compute_image_mean generate mean file from lmdb files in 2) (success) but leveldb files in 1) (fail).

[Ronak Gupta]

@Zeng:  have you solved the problem to generate leveldb files for CIFAR-10 dataset?

I got four empty files (lock, log, and log.old, MANIFEST) and no .sst files.

When I try to read an already created leveldb files, I get following error:

$ ./train_full.sh
    I0622 00:04:40.815856 28860 train_net.cpp:26] Starting Optimization
    I0622 00:04:40.816721 28860 solver.cpp:26] Creating training net.
    I0622 00:04:40.816889 28860 net.cpp:70] Creating Layer data
    I0622 00:04:40.816900 28860 net.cpp:105] data -> data
    I0622 00:04:40.816915 28860 net.cpp:105] data -> label
    I0622 00:04:40.816928 28860 data_layer.cpp:148] Opening leveldb cifar10_gcn_padded-leveldb/cifar-train-leveldb
    F0622 00:04:40.816994 28860 data_layer.cpp:151] Check failed: status.ok() Failed to open leveldb cifar10_gcn_padded-leveldb/cifar-train-leveldb
    IO error: cifar10_gcn_padded-leveldb/cifar-train-leveldb/LOCK: No such file or directory

    *** Check failure stack trace: ***

        @     0x7fec1db18b7d  google::LogMessage::Fail()
        @     0x7fec1db1ac7f  google::LogMessage::SendToLog()
        @     0x7fec1db1876c  google::LogMessage::Flush()
        @     0x7fec1db1b51d  google::LogMessageFatal::~LogMessageFatal()
        @           0x45d17f  caffe::DataLayer<>::SetUp()
        @           0x43a434  caffe::Net<>::Init()
        @           0x43bb2a  caffe::Net<>::Net()
        @           0x426c7c  caffe::Solver<>::Solver()
        @           0x40e79f  main
        @     0x7fec1b8607ed  (unknown)
        @           0x40fdcd  (unknown)
    Aborted (core dumped)