Data reshape for fully connected layers

[naranjuelo]

Hi! In fully connected layers (inner product) we have the input data of N*C*W*H dimensions, and before multiplying it by the weights of the layer, data is reshaped into a single vector. As it is explained in inner_product_layer.cpp :

// Dimensions starting from "axis" are "flattened" into a single

// length K_ vector. For example, if bottom[0]'s shape is (N, C, H, W),

// and axis == 1, N inner products with dimension CHW are performed.

But how is organized this "flattened" vector? How does caffe create it?
For example, does it start with the first input channel's first row and continues with its following rows, and repeats it with the C channels? Thank you!!

[Przemek D]

Bumping as this is an interesting question.
I don't think inner_product_layer.cpp is a good place to look for this - after all, all that it does is creates a new layer of dimensions N*1*1*K without bothering about data in its bottom blob. After all, each of the K elements of a vector will be computed from all C*W*H elements of the input volume, so the order doesn't matter here. I suppose reshape_layer.cpp would contain more info on this, but I have a hard time understanding what's going on in there...

[naranjuelo]

Hi, thank you for your suggestion! Actually, I have discovered how they order it. It starts with first channel's first row's first element, then second element in the same row... until completing the row. Next same thing with second row and so on until completing the first channel. Next the second channel is done, third one... until C channels.

[Przemek D]

So, to sum up: if we reshape a N*C*W*H blob into a N*1*1*K vector, our traverse priority is H (3) dimension first, then W (2), and finally C (1), for a total of N (0) times?
Mark the thread as answered (closed) if you solved it :)

[naranjuelo]

W, H and finally C, for a total of N times, that's it