Normalizing hidden layer output to range -1 to 1

[Bharat Bhusan Sau]

Hi,

I need to normalize my data(some hidden layer output) to range [-1,+1] , i.e. x = [ { x - min(x) }*2 / { max(x) - min(x) } ] - 1  .

I could not find any layer or combination of layers present in caffe. 

If anybody knows how to do it, please reply.

Thanks in advance.

[Przemek D]

You could try writing a custom Python layer. Here's a nice example on how to do that.

[Atena Nguyen]

Do you mean all hidden nodes at a particular layer? 

In my opinion, you can make it with a combination of maxpooling, minpooling, scale and concatination layer. 

Vào 05:36:36 UTC+9 Thứ Sáu, ngày 04 tháng 8 năm 2017, Bharat Bhusan Sau đã viết:

[Przemek D]

In my opinion, this would be an unnecessarily complex origami of layers to achieve something that a single Python layer would in a couple lines of code.
Let's see... you need pooling to get max(), then a dummy data, eltwise and another pooling to get min() - Caffe does not support min pooling directly, afaik, so you need to work around that by negating a copy of the blob. This is already 4 layers to only get the min&max, and there's still a fairly complex equation to build, with 5 operations (3 subtractions, sqr and division) and another constant. Unless I fail to see a simpler way to do that using layers, that's a huge hassle. Worst of all, if he finds that his normalization works poorly and decides to change it... redesigning this using layers will give him another headache, while changing an existing Python layer will take a minute ;)

[Jonathan R. Williford]

Bharat,

You can normalize the data to have mean 0 and variance 1 fairly easily with the MVN layer. You could scale the result to reduce the variance more, such that very few examples fall outside of the range of -1 to 1.

A custom python layer may slow down training, if you are using GPU.

Why do you want to do this? There might be a better solution, if we know what you are trying to accomplish.

Cheers,

Jonathan

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users+unsubscribe@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/e15e0da9-9a29-404a-b56a-43e264925542%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Atena Nguyen]

Hi Przemek D, 

Your suggestion is reasonable if we working on python only environment. I wonder the case when we want to deploy the network in C++. In that case, I have to make c++ code for that python layer too? 

Btw, could you provide some useful link for python layer?, 

Best regards, 

Vào 19:18:14 UTC+9 Thứ Tư, ngày 23 tháng 8 năm 2017, Przemek D đã viết:

[Przemek D]

If you want to work in a pure C++ environment then you will be better off writing your own layer. I'm speaking more generally here, not knowing what you want to do - some tasks can be achieved efficiently with combinations of existing layers. A few links:
- guide to developing python layers
- introduction to developing C++ layers
- example of a C++ layer development