: "examples/imagenet/create_imagenet.sh", setting RESIZE=true and RESIZE_WIDTH=256, RESIZE_HEIGHT=256.
Now i am trying to replicate the behavior of the create_imagenet.sh script manually using python with opencv. I need these manually generated images to perform some further experiments on the training/validation sets. Below is a sample of the python script i'm using to test the manual generation:
import numpy as np
import re
import os
import cv2
import caffe
lmdb_env = lmdb.open('../../tools/caffe/examples/imagenet/ilsvrc12_train_lmdb')
files_basename = '../../../../imagenet/ilsvrc2012_train/'
print("Test: read image from LMDB and compare with corresponding file (after resizing)")
count_diff = 0
with lmdb_env.begin() as lmdb_transaction:
cursor = lmdb_transaction.cursor()
for key, value in cursor:
datum = caffe.proto.caffe_pb2.Datum()
datum.ParseFromString(value)
imlmdb = caffe.io.datum_to_array(datum)
# move color channel at the end
imlmdb = np.moveaxis(imlmdb, 0, -1)
# convert lmdb key to path
filename = key.decode('ascii').split('_', 1)[-1]
path = os.path.join(files_basename, filename)
imfile_orig = cv2.imread(path)
imfile = cv2.resize(imfile_orig, imlmdb.shape[:-1], interpolation = cv2.INTER_LINEAR)
# convert to int32 to have signed difference
maxdiff = np.max(np.abs(imlmdb.astype('int32') - imfile.astype('int32')))
if maxdiff > 0:
count_diff += 1
print('[WARNING] Found difference. Max. diff.:', maxdiff, ', total count:', count_diff)
cv2.imshow('lmdb', imlmdb)
cv2.imshow('file', imfile)
cv2.waitKey(0)
cv2.destroyAllWindows()
if count_diff == 0:
print("[OK] No difference found")
The result i get is that about 200k images over the 1.2M of the ImageNet training set are not identical when i retrieve them from LMDB compared to when i read them manually from file and resize them. The maximum difference over all pixel's color components is in the order of 5 to 20.
I checked the code in 'caffe/tools/convert_imageset.cpp' and in 'caffe/src/caffe/util/io.cpp' (functions 'ReadImageToDatum' and 'ReadImageToCVMat') but i couldn't find any apparent reason why the two versions should differ. I thought it could be a problem of interpolation during resizing, but 'cv::resize()' should use bilinear interpolation by default, and i am enforcing that algorithm in my script.
Obviously, both create_imagenet.sh and my python script are using the same opencv version (2.4.13 compiled from source on CentOS 6)
Do you have any idea on how i can fix this, so that i can recreate the training set data exactly in python?
Thanks very much in advance.
DJP.