Help debugging difference between model.predict() and model.predict_generator() for same image

[gz]

Hello,

I'm following the transfer learning tutorial here (https://blog.keras.io/building-powerful-image-classification-models-using-very-little-data.html) to create bottleneck features (output of fifth max pooling layer of VGG16) for a large number of images. I'm using the ImageDataGenerator with the following settings:

datagen = ImageDataGenerator(featurewise_center=True)
datagen.mean = np.array([103.939, 116.779, 123.68],dtype=np.float32).reshape(3,1,1)
generator = datagen.flow_from_directory(
        train_data_dir,
        target_size=(224, 224),
        batch_size=32,
        class_mode=None,
        shuffle=False)
bottleneck_features_train = model.predict_generator(generator, generator.N)

I have one image in my training directory, which is already of size 224x224. The code works fine, and I get a tensor out of size (1, 512, 7, 7).

Before chancing upon that blog post and using that code, I was computing the activations by simply using the model.predict() function. Here is my code for that process, which I assume (correct me if I'm wrong) should give the same result:

import cv2

def load_img(imgpath):
    img = cv2.imread(imgpath)
    im = img.astype(np.float32)
    im[:,:,0] -= 103.939
    im[:,:,1] -= 116.779
    im[:,:,2] -= 123.68
    # change indices to [channel, height, width]
    imt = im.transpose((2,0,1))
    imx = np.expand_dims(imt, axis=0)
    return imx

output = model.predict(load_img(train_data_img_path))

In this case, train_data_img_path is the same image (and the only image) in the train_data_dir directory. The output I get is a tensor of the same shape as bottleneck_features_train as well. However, the values are different!

In light of this, I'm wondering, am I doing something in the constructor of the ImageDataGenerator that is incorrect? Am I choosing a default setting that is doing something other than just subtracting the featurewise mean? I looked through the source code of ImageDataGenerator, and I can't figure out if there's anything else it does. I don't think it carries out any other transformations on the image. 

My next step is to use the ImageDataGenerator's ability to output debug images and check there to see what is going on, but I figured I'd post now just to see if anyone can spot an error or discrepancy between the transformations in my code. 

Just in case it's relevant, I'm using TensorFlow with Theano image dimension ordering.

Thank you in advance! I'll be back tomorrow (Pacific US time).

[Vladimir Iglovikov]

Do both codes give the same result if you comment line 

imt = im.transpose((2,0,1))

?

[Qixianbiao Qixianbiao]

cv2 read image in BGR mode, but keras.preprocessing use "from PIL import Image" which reads images in "RGB"(please see the code). This is different. So, you should take care of this..

@Valdimir IgIovikov, 

imt = im.transpose((2,0,1))   is used to adopt the data into theano format, it should be there or it will raise error.

在 2016年8月5日星期五 UTC+8上午10:50:58，gz写道：

[ahtesham...@gmail.com]

Can you please explain how did you calculate the following mean values [103.939, 116.779, 123.68]?

[gz]

These mean values were calculated from the original VGG model's training set. I believe they were published by the authors of the VGG paper. See e.g. the following for some background:

• https://gist.github.com/fchollet/f35fbc80e066a49d65f1688a7e99f069
  
• https://gist.github.com/baraldilorenzo/07d7802847aaad0a35d3