Another error with create_imagenet.sh

[Fernando Camacho]

Hello,
I am running with problems when trying to create lmbd files for training and validation. I found a similar problem to mine, but the solution given there does not apply to my case since I already have an empty directory for the lmbd files. Further more the problem appeared after I update my computer, previously I had been able to create lmbd files.
I am using Ubuntu 14.04, any help would be greatly appreciated.

Here is the error message

Creating train lmdb...
I0529 14:33:24.970546  3536 convert_imageset.cpp:79] Shuffling data
I0529 14:33:24.971197  3536 convert_imageset.cpp:82] A total of 187 images.
F0529 14:33:24.971287  3536 db.cpp:27] Check failed: mkdir(source.c_str(), 0744) == 0 (-1 vs. 0) mkdir /home/fernando/Documents/math/Summer-2015/DeepLearning/caffeExample/lmdbFile/houseStyle_train_lmdbfailed
*** Check failure stack trace: ***
    @     0x7ff7a6f18daa  (unknown)
    @     0x7ff7a6f18ce4  (unknown)
    @     0x7ff7a6f186e6  (unknown)
    @     0x7ff7a6f1b687  (unknown)
    @     0x7ff7a725f84e  caffe::db::LMDB::Open()
    @           0x403122  main
    @     0x7ff7a5ce5ec5  (unknown)
    @           0x403e5c  (unknown)
    @              (nil)  (unknown)
create_imagenet.sh: line 45:  3536 Aborted                 (core dumped) GLOG_logtostderr=1 $TOOLS/convert_imageset --resize_height=$RESIZE_HEIGHT --resize_width=$RESIZE_WIDTH --shuffle $TRAIN_DATA_ROOT $DATA/train.txt $EXAMPLE/houseStyle_train_lmdb
Creating val lmdb...
I0529 14:33:25.211222  3538 convert_imageset.cpp:79] Shuffling data
I0529 14:33:25.211961  3538 convert_imageset.cpp:82] A total of 34 images.
F0529 14:33:25.212079  3538 db.cpp:27] Check failed: mkdir(source.c_str(), 0744) == 0 (-1 vs. 0) mkdir /home/fernando/Documents/math/Summer-2015/DeepLearning/caffeExample/lmdbFile/houseStyle_val_lmdbfailed
*** Check failure stack trace: ***
    @     0x7fe302aeedaa  (unknown)
    @     0x7fe302aeece4  (unknown)
    @     0x7fe302aee6e6  (unknown)
    @     0x7fe302af1687  (unknown)
    @     0x7fe302e3584e  caffe::db::LMDB::Open()
    @           0x403122  main
    @     0x7fe3018bbec5  (unknown)
    @           0x403e5c  (unknown)
    @              (nil)  (unknown)
create_imagenet.sh: line 55:  3538 Aborted                 (core dumped) GLOG_logtostderr=1 $TOOLS/convert_imageset --resize_height=$RESIZE_HEIGHT --resize_width=$RESIZE_WIDTH --shuffle $VAL_DATA_ROOT $DATA/val.txt $EXAMPLE/houseStyle_val_lmdb
Done.

[Antonio Paes]

Hi, you erased the directorys before you run the create script?

If yes, your txt of images are labeled correct?

how is a type o your images?

[Fernando Camacho]

Antonio,
Thank you, I found the problem in the end it was a silly mistake. The name of my new directory didn't match the one I entered in create_imagenet.sh. In create_imagenet.sh I set
EXAMPLE=/home/fernando/Documents/math/Summer-2015/DeepLearning/caffeExample/lmdbFile
, but the directory I created was
/home/fernando/Documents/math/Summer-2015/DeepLearning/caffeExample/mdbFile.
I was missing the "l" at the begining of lmdbFile.

On Saturday, May 30, 2015 at 9:12:46 AM UTC-4, Antonio Paes wrote:

Hi, you erased the directorys before you run the create script?

If yes, your txt of images are labeled correct?

how is a type o your images?

Em sexta-feira, 29 de maio de 2015 16:23:53 UTC-3, Fernando Camacho escreveu:

Hello,
I am running with problems when trying to create lmbd files for training and validation. I found a similar problem to mine, but the solution given there does not apply to my case since I already have an empty directory for the lmbd files. Further more the problem appeared after I update my computer, previously I had been able to create lmbd files.
I am using Ubuntu 14.04, any help would be greatly appreciated.

Here is the error message

Creating train lmdb...
I0529 14:33:24.970546  3536 convert_imageset.cpp:79] Shuffling data
I0529 14:33:24.971197  3536 convert_imageset.cpp:82] A total of 187 images.

F0529 14:33:24.971287  3536 db.cpp:27] Check failed: mkdir(source.c_str(), 0744) == 0 (-1 vs. 0) mkdir /home/fernando/Documents/math/Summer-2015/DeepLearning/caffeExample/houseStyle_train_lmdbfailed

[Carlos Treviño]

Hi Fernando!

I'm also trying to generate my own lmdb files, but even with the original create_imagenet.sh file i get this error:

tools/convert_imageset: No such file or directory

Any idea?
-Carlos

[Antonio Paes]

Hi you compile caffe?

[Carlos Treviño]

Hi,

Yes, I did. I was calling the .cpp file instead of the .bin file. but still now i get the next message:

vpn26055:~ CarlosTrevino$ #!/usr/bin/env sh#!/usr/bin/env sh

vpn26055:~ CarlosTrevino$ # Create the camvid lmdb inputs

vpn26055:~ CarlosTrevino$ # N.B. set the path to the camvid train + val data dirs

vpn26055:~ CarlosTrevino$ cd caffe_win_01062015

vpn26055:caffe_win_01062015 CarlosTrevino$ EXAMPLE=examples/camvid

vpn26055:caffe_win_01062015 CarlosTrevino$ DATA=data/camvid/rawpixels

vpn26055:caffe_win_01062015 CarlosTrevino$ TOOLS=build/tools

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ TRAIN_DATA_ROOT=data/camvid/rawpixels

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ # Set RESIZE=true to resize the images to 256x256. Leave as false if images have

vpn26055:caffe_win_01062015 CarlosTrevino$ # already been resized using another tool.

vpn26055:caffe_win_01062015 CarlosTrevino$ RESIZE=true

vpn26055:caffe_win_01062015 CarlosTrevino$ if $RESIZE; then

>   RESIZE_HEIGHT=360

>   RESIZE_WIDTH=480

> else

>   RESIZE_HEIGHT=0

>   RESIZE_WIDTH=0

> fi

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ if [ ! -d "$TRAIN_DATA_ROOT" ]; then

>   echo "Error: TRAIN_DATA_ROOT is not a path to a directory: $TRAIN_DATA_ROOT"

>   echo "Set the TRAIN_DATA_ROOT variable in create_camvid.sh to the path" \

>        "where the Camvid training data is stored."

>   exit 1

> fi

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ echo "Creating train lmdb..."

Creating train lmdb...

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ GLOG_logtostderr=1 $TOOLS/convert_imageset \

>     --resize_height=$RESIZE_HEIGHT \

>     --resize_width=$RESIZE_WIDTH \

>     --shuffle \

>     $TRAIN_DATA_ROOT \

>     $DATA/train.txt \

>     $EXAMPLE/camvid_train_lmdb

echo "Done."

I0603 13:31:48.530912 1968370432 convert_imageset.cpp:79] Shuffling data

I0603 13:31:48.537243 1968370432 convert_imageset.cpp:82] A total of 0 images.

I0603 13:31:48.550928 1968370432 db.cpp:34] Opened lmdb examples/camvid/camvid_train_lmdb

vpn26055:caffe_win_01062015 CarlosTrevino$ 

vpn26055:caffe_win_01062015 CarlosTrevino$ echo "Done."

Done.

vpn26055:caffe_win_01062015 CarlosTrevino$ 

Any advice will be appreciated.

-Carlos

[Antonio Paes]

Hi Carlos, you've certainty which directories of train and val are correct?

I've this problem some time ago, and i note which I was going the wrong way.

[Carlos Treviño]

It seems that i'm getting closer to the solution, but i'm not able to read the images

Creating train lmdb...

I0603 16:49:02.657023 1968370432 convert_imageset.cpp:79] Shuffling data

I0603 16:49:02.662811 1968370432 convert_imageset.cpp:82] A total of 1 images.

I0603 16:49:02.663348 1968370432 db.cpp:34] Opened lmdb examples/camvid/example_train_lmdb

E0603 16:49:02.663455 1968370432 io.cpp:77] Could not open or find file data/camvid/trainUsers/CarlosTrevino/caffe_win_01062015/data/camvid/train/gato.jpg

Done.

[Antonio Paes]

check the images and txt files.

[Swami]

Has anyone here solved their problem ? I am having the same issue. I checked that the file exists and thatit can be read by other apps like Matlab and a stand alone opencv code too.

[Sajjad Behravesh]

Hi
I have a problem with create_imagenet.sh to create lmbd files, when set flag of Resize = true. following error is occured. how to fix this ?

./create_imagenet.sh
Creating train lmdb...
I0904 05:59:11.694593 11347 convert_imageset.cpp:86] Shuffling data
I0904 05:59:11.905416 11347 convert_imageset.cpp:89] A total of 662 images.
I0904 05:59:11.905696 11347 db_lmdb.cpp:35] Opened lmdb /home/deeplearning/Develop/caffe/data/iranshahr/ilsvrc12_train_lmdb
Segmentation fault (core dumped)

[xiaodan....@gmail.com]

在 2016年9月4日星期日 UTC+8上午9:32:36，Sajjad Behravesh写道：

[xiaodan....@gmail.com]

I have the same problem. Have you found the reason? How to solve it?

在 2016年9月4日星期日 UTC+8上午9:32:36，Sajjad Behravesh写道：

Hi