I cannot duplicate the accuracy of VGG16 on ImageNet Validation Set

[xinc...@gmail.com]

Hello everyone, 

I am trying to evaluate the pre-trained VGG16 model on ImagenNet validation set. (keras 2.1.6, Tensorflow-gpu 1.8.0)

However, I cannot duplicate the accuracy reported on Keras’ documents.

The following shows what I did:

    1. Download ImageNet validation set using https://github.com/tensorflow/models/blob/master/research/slim/datasets/download_imagenet.sh

    2. Download ground truth for validation from https://github.com/tensorflow/models/blob/master/research/slim/datasets/imagenet_2012_validation_synset_labels.txt

    3. Move validation images to the class folder using https://github.com/awslabs/deeplearning-benchmark/blob/master/tensorflow/inception/inception/data/preprocess_imagenet_validation_data.py

    4. Use this code to evaluate the accuracy of VGG16 on ImageNet validation.

I only got 0.63(top1) and 0.85(top5) accuracy. 

However, Keras claims that they achieve 0.715(top1) and 0.901(top5) on ImageNet validation set.

Did I make any mistake? 

Thank you very much. 

Sincerely,

Xin

#######

from keras.applications import VGG16

from keras.applications.vgg16 import preprocess_input

from keras.preprocessing.image import ImageDataGenerator

from keras import metrics

val_datagen = ImageDataGenerator(preprocessing_function=preprocess_input)

validation_generator = val_datagen.flow_from_directory(

'/data/xincoder/imagenet/imagenet-datavalidation',

target_size=(224, 224), 

batch_size=10,

class_mode='categorical',

shuffle=False)

model = VGG16(weights='imagenet', include_top=True)

model.trainable=False

model.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['acc', metrics.top_k_categorical_accuracy])

results = model.evaluate_generator(validation_generator, steps=5000, workers=1, max_queue_size=1)

print(results)

#######

[ken.xi...@gmail.com]

Hi,  

Wonder if you have found the problem but I am getting the same results as yours. Not sure what have I done wrong here ...

Best,

K