Hdf5 - ValueError: design_matrix must have 2 dimensions

[mathieu...@gmail.com]

Hi,

I am experiencing some problems using pylearn2 with hdf5 format for my data.
My code is based on multilayer_perceptron tutorial and adapted to my input, but it returns the following error :

Traceback (most recent call last):
File "./main.py", line 24, in <module>
train.main_loop()
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/train.py", line 201, in main_loop
extension_continue = self.run_callbacks_and_monitoring()
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/train.py", line 255, in run_callbacks_and_monitoring
self.model.monitor()
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/monitor.py", line 254, in __call__
for X in myiterator:
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/datasets/hdf5_deprecated.py", line 260, in next
this_data = fn(this_data)
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/datasets/dense_design_matrix.py", line 292, in <lambda>
self.view_converter.get_formatted_batch(batch, space))
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/datasets/dense_design_matrix.py", line 1474, in get_formatted_batch
topo_batch = self.design_mat_to_topo_view(batch)
File "/home/mathieu/Bureau/semestre8/PAO/pylearn2_codeSource/pylearn2/datasets/dense_design_matrix.py", line 1392, in design_mat_to_topo_view
"was %s." % str(design_matrix.shape))
ValueError: design_matrix must have 2 dimensions, but shape was (100, 480, 640, 1).

I have changed the train_iteration_mode parameter and modified the compute_test_err.py file as advised on other topics, but without success. Some help would be very appreciated.

My data :
HDF5 "pongsplitpao-b01c.hdf5" {
DATASET "/train/X" {
DATATYPE H5T_STD_U8LE
DATASPACE SIMPLE { ( 5250, 480, 640, 1 ) / ( 5250, 480, 640, 1 ) }
...
}
}

My code :
!obj:pylearn2.train.Train {

dataset: &train !obj:pylearn2.datasets.hdf5.HDF5Dataset {
filename: '/media/mathieu/DoublePerso/Documents/data/NaoPong/pongsplitpao-b01c.hdf5',
X: train/X,
y: train/y,
view_converter: !obj:pylearn2.datasets.dense_design_matrix.DefaultViewConverter {
shape: [480, 640, 1],
axes: ['b',0, 1, 'c']
}
},

model: !obj:pylearn2.models.mlp.MLP {
layers: [!obj:pylearn2.models.mlp.ConvRectifiedLinear {
layer_name: 'h0',
output_channels: 11,
irange: .05,
kernel_shape: [2, 2],
pool_shape: [2, 2],
pool_stride: [2, 2],
},
!obj:pylearn2.models.mlp.RectifiedLinear {
layer_name: 'h1',
dim: %(dim_h1)i,
sparse_init: 15
}, !obj:pylearn2.models.mlp.RectifiedLinear {
layer_name: 'h2',
dim: %(dim_h2)i,
sparse_init: %(sparse_init_h2)i
}, !obj:pylearn2.models.mlp.Linear {
layer_name: 'y',
dim: 4,
sparse_init: %(sparse_init_h2)i
}
],
input_space: !obj:pylearn2.space.Conv2DSpace {
shape: [480, 640],
num_channels: 1,
axes: ['b',0,1,'c']
}
},

algorithm: !obj:pylearn2.training_algorithms.sgd.SGD {
batch_size: 100,
train_iteration_mode: 'batchwise_shuffled_sequential',
learning_rate: .05,
monitoring_dataset: {
'train' : *train,
'valid' : !obj:pylearn2.datasets.hdf5.HDF5Dataset {
filename: '/media/mathieu/DoublePerso/Documents/data/NaoPong/pongsplitpao-b01c.hdf5',
X: valid/X,
y: valid/y,
view_converter: !obj:pylearn2.datasets.dense_design_matrix.DefaultViewConverter {
shape: [480,640,1],
axes: ['b', 0, 1, 'c']
}
},
'test' : !obj:pylearn2.datasets.hdf5.HDF5Dataset {
filename: '/media/mathieu/DoublePerso/Documents/data/NaoPong/pongsplitpao-b01c.hdf5',
X: test/X,
y: test/y,
view_converter: !obj:pylearn2.datasets.dense_design_matrix.DefaultViewConverter {
shape: [480, 640, 1],
axes: ['b', 0, 1, 'c']
}
},
},
learning_rule: !obj:pylearn2.training_algorithms.learning_rule.Momentum {
init_momentum: .5
},
termination_criterion: !obj:pylearn2.termination_criteria.And {
criteria: [
!obj:pylearn2.termination_criteria.MonitorBased {
channel_name: "valid_y_mse",
prop_decrease: 0.1,
N: 10
},
!obj:pylearn2.termination_criteria.EpochCounter {
max_epochs: %(max_epochs)i
}
]
},
},

}