nnet.blobs['data'].reshape(1,3,h,w)
nnet.transformer = caffe.io.Transformer({'data': nnet.blobs['data'].data.shape})
#Print the layer blob shape
for k,v in net.blobs.items():
print (k, v.data.shape)
Then you should be able to simple resize your segmentation ground truth, to the size of last convolutional layer of your network, right?