Difficulty when train alexnet on imagenet myself

[X.H. Jiang]

I want to train an alexnet(bvlc_ref) model on imagenet(ILSVRC2012) dataset from scratch all by myself.

But I tried for serval times, the accuracy kept very small, and the loss significantly big.

Till now my max acc is 0.0370899, my min loss is 5.74959.

The training has processed 274800 iterations.

I do this according to steps in http://caffe.berkeleyvision.org/gathered/examples/imagenet.html

My command sequence here:

export IMAGENET_PATH=~/images/imagenet2012

cd $CAFFE_ROOT

vi ./data/ilsvrc12/get_ilsvrc_aux.sh

./data/ilsvrc12/get_ilsvrc_aux.sh

vi examples/imagenet/create_imagenet.sh

    TRAIN_DATA_ROOT=$IMAGENET_PATH/train/

    VAL_DATA_ROOT=$IMAGENET_PATH/val/

    RESIZE=true

rm -rf examples/imagenet/ilsvrc12_train_lmdb

rm -rf examples/imagenet/ilsvrc12_val_lmdb

./examples/imagenet/create_imagenet.sh

./examples/imagenet/make_imagenet_mean.sh

./build/tools/caffe train --solver=models/bvlc_reference_caffenet/solver.prototxt

My dataset:

data/ilsvrc12/train.txt

    1281167L

data/ilsvrc12/val.txt

    5wL

    

My solver:

cat models/bvlc_reference_caffenet/solver.prototxt

net: "models/bvlc_reference_caffenet/train_val.prototxt"

test_iter: 256

test_interval: 5005

base_lr: 0.0001

lr_policy: "step"

gamma: 0.1

stepsize: 50050

display: 20

max_iter: 500500

momentum: 0.9

weight_decay: 0.0005

snapshot: 5005

snapshot_prefix: "models/bvlc_reference_caffenet/caffenet_train"

solver_mode: GPU

My batch_sizes:

train: 256

test: 200

Other solver and train_val params keep not touched. 

Seems I'm stucked here again.

Has anyone trained imagenet by self successfully?

Can you share your super params here?

And anyone would tell me what's wrong in my run steps or params?

Thank you very much!