Batch size and interations, am I doing it correctly?

227 views
Skip to first unread message

Hossein Hasanpour

unread,
Mar 12, 2016, 7:35:13 AM3/12/16
to Caffe Users
Hello All,
I am working on CIFAR10 dataset and experimenting with it. I'm however, confused about batch-size and iteration.
In the solver file provided by caffe, for cifar10 example, we have these parameters :

cifar10_full_solver:
test_iter: 100
test_interval
: 1000
# The maximum number of iterations
max_iter
: 60000

cifar10_full_train_test.prototxt

name: "CIFAR10_full"
layer
{
  include
{
    phase
: TRAIN
 
}
   data_param
{
     batch_size
: 100
 
}
}
layer
{
   include
{
    phase
: TEST
 
}
  data_param
{
     batch_size
: 100
 
}
}

So we have 60K training images right? and we have batch size of 100 + 60K iterations, so basically 100*60K = 6000K and that is 100 times the number of our training set!
Does this mean, we have 100 epochs? we basically reiterate our training set 100 times right? Is it Ok? Is it right to do so ?
and about the testing phase, 100*100 equals 10,000 and each 1000 iteration, we test the network . so if I ever need to change the batch size by any factor, I need to multiply the test_iteration by that factor as well right ?
Now what about this configuration ?

test_iter: 20
test_interval
: 1000
# The maximum number of iterations
max_iter
: 60000
 
# The learning rate policy
type
:"AdaDelta"
delta
:1e-5
gamma
: 0.1    
lr_policy
: "multistep"
stepvalue
: 12000
stepvalue
: 18000
stepvalue
: 25000
stepvalue
: 35000
stepvalue
: 45000
stepvalue
: 55000

name: "CIFAR10_full"
layer
{
  include
{
    phase
: TRAIN
 
}
  data_param
{
    batch_size
: 50#100
 
}
}
layer
{
  include
{
    phase
: TEST
 
}
  data_param
{
    batch_size
: 500#1000
 
}
}

Is there anything wrong with this configuration ?
What happens if I lower the batch size for test even more?
what happens if I lower the batch size for training ? for example set it to 10! and make max_iteration like a 60M?!!
does this affect the accuracy of the network in a bad way? or make the network give wrong results ( say it says the accuracy is 90% while this 90% is flawed is its possible? )


I'd be grateful to know if I'm doing something wrong here.

Nam Vo

unread,
Mar 13, 2016, 3:58:29 PM3/13/16
to Caffe Users
I don't see anything wrong.
You just have to run and see what works.
Reply all
Reply to author
Forward
0 new messages