Fwd: Email from Kaggle User 'José A. Guerrero'

[Ian Goodfellow]

Here's another problem with the yaml parser: if you specify the same
keyword argument twice, one of the arguments just gets ignored. In
this case it means the MomentumAdjustor gets ignored. To be consistent
with python, we should raise a SyntaxError in this situation.


---------- Forwarded message ----------
From: Kaggle Administrator <sup...@kaggle.com>
Date: Sat, May 4, 2013 at 9:33 AM
Subject: Email from Kaggle User 'José A. Guerrero'
To: goodfel...@gmail.com


This introductory contact email was sent via the Contact User button
on your Kaggle user profile.
Please DO NOT reply directly to this email, you should contact sender
at their address below.

If you feel this email is inappropriate please report abuse here.
To block all future contact attempts by all Kaggle users, please check
the "Block all user contact emails" preference in your user profile
here.

From: José A. Guerrero <jague...@ono.com> Kaggle Profile

Subject: MomentumAdjustor

Hi, Ian
Thank you for your help.

!obj:pylearn2.train.Train {
# Here we specify the dataset to train on. We train on only the
first 900 of the examples, so
# that the rest may be used as a validation set.
# The "&train" syntax lets us refer back to this object as
"*train" elsewhere in the yaml file
dataset: &train
!obj:pylearn2.scripts.icml_2013_wrepl.black_box.black_box_dataset.BlackBoxDataset
{
which_set: 'train',
start: 100,
stop: 1000
},
# Here we specify the model to train as being an MLP
model: !obj:pylearn2.models.mlp.MLP {
batch_size: 25,
layers : [
# We use two hidden layers with rectified linear activations
!obj:pylearn2.models.mlp.RectifiedLinear {
layer_name: 'h0',
dim: 1875,
irange: .05,
# Rather than using weight decay, we constrain the
norms of the weight vectors
max_col_norm: 2.
},
!obj:pylearn2.models.mlp.RectifiedLinear {
layer_name: 'h1',
dim: 1875,
irange: .05,
# Rather than using weight decay, we constrain the
norms of the weight vectors
max_col_norm: 2.
},
!obj:pylearn2.models.mlp.Softmax {
layer_name: 'y',
init_bias_target_marginals: *train,
# Initialize the weights to all 0s
irange: .0,
n_classes: 9
}
],
nvis: 1875,
},
# We train using SGD and momentum
algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
learning_rate: .10,
init_momentum: .3,
# We monitor how well we're doing during training on a validation set
monitoring_dataset:
{
'train' : *train,
'valid' :
!obj:pylearn2.scripts.icml_2013_wrepl.black_box.black_box_dataset.BlackBoxDataset
{
which_set: 'train',
start: 0,
stop: 100,
}
},
# We stop when validation set classification error hasn't
decreased for 10 epochs
#termination_criterion:
!obj:pylearn2.termination_criteria.EpochCounter {
# max_epochs: 30},
termination_criterion: !obj:pylearn2.termination_criteria.MonitorBased {
channel_name: "valid_y_nll",
prop_decrease: 0.,
N: 30
},
update_callbacks: [
# This callback makes the learning rate shrink by
dividing it by decay_factor after
# each sgd step.
!obj:pylearn2.training_algorithms.sgd.ExponentialDecay {
decay_factor: 1.0025,
min_lr: 0.005
}
],
},
extensions: [
!obj:pylearn2.training_algorithms.sgd.MomentumAdjustor {
start: 2,
saturate: 50,
final_momentum: .7
}
],
# We save the model whenever we improve on the validation set
classification error
extensions: [
!obj:pylearn2.train_extensions.best_params.MonitorBasedSaveBest {
channel_name: 'valid_y_misclass',
save_path: "${PYLEARN2_TRAIN_FILE_FULL_STEM}_best.pkl"
},
],
save_path: "mlp.pkl",
save_freq: 5
}

[Ian Goodfellow]

I didn't write the HigherOrderContractiveAutoencoder but if you look at

scripts/autoencoder_example/hcae.yaml

it looks like you have to manually add the higher order term to the cost.

On Fri, May 10, 2013 at 6:57 PM, José A. Guerrero <jague...@ono.com> wrote:

Hi Ian

 

Thank you very much for your support and patience in this challenge.

 

Using only CPU, GaussianCorruptor with stdev=.3 and stdev=.5 give me exactly the same trace as you seen in screenshot.

 

Any tip?