Can't set Solver to Adam?

[Alex]

Hello,

so here's my solver.prototxt:

net: "models/dcnf-fcsp-alexnet/train_val.prototxt"
#test_iter: 1000
#test_interval: 1000
test_initialization: false
base_lr: 0.0001
lr_policy: "step"
gamma: 0.01
stepsize: 50000
display: 20
max_iter: 1000000
momentum: 0.9
weight_decay: 0.0005
snapshot: 10000
snapshot_prefix: "/data/lesi/dcnf-fcsp-alexnet/"
type: "Adam"
solver_mode: GPU

It cleary shoud set the type to Adam. Yet when I run a training with this solver my supervisor pointed out that it looks like it's using SGD (since it says sdg_solver.cpp):

I0728 16:18:59.490665 27998 sgd_solver.cpp:106] Iteration 41860, lr = 0.0001
I0728 16:19:26.414223 27998 solver.cpp:228] Iteration 41880, loss = 1.45618
I0728 16:19:26.414342 27998 solver.cpp:244]     Train net output #0: loss = 1.45618 (* 1 = 1.45618 loss)
I0728 16:19:26.414355 27998 sgd_solver.cpp:106] Iteration 41880, lr = 0.0001
I0728 16:19:53.348322 27998 solver.cpp:228] Iteration 41900, loss = 1.44106
I0728 16:19:53.348362 27998 solver.cpp:244]     Train net output #0: loss = 1.44106 (* 1 = 1.44106 loss)

Is this just some console output confusion or am I actually using SGD? If so, how come it won't switch to Adam?

[Atena Nguyen]

How did you run the training? using python or command line?

Vào 23:57:39 UTC+9 Thứ Sáu, ngày 28 tháng 7 năm 2017, Alex đã viết:

[Przemek D]

Since this question has been answered on Stack Overflow, here's a link to Shai's explanation: https://stackoverflow.com/a/45396522