How can I create my new type of python loss layer with parameters on python?

[Peerajak Witoonchart]

I am creating new type of loss function with my own weights and parameters for this loss function.


I would like to do

1. Forward a batch of training samples on GPU
2. get the output blobs and then calculate my own loss value, and gradient of the loss on CPU
( I have my own way of loss, and gradient calculation )
3. back propagate the gradient given by 2 to the rest of the network on GPU
4. Update all layer parameters according to gradient in 3.
5. Once a while save the learned network parameters.


Can I do this on python?  I read this https://github.com/BVLC/caffe/issues/1855  but not yet see how my wish can be implemented

many thanx

[MahAbb]

I would like to know about defining a new loss function for the loss layer in Deep Net. Any help appreciate!

Thank you

[Evan Shelhamer]

A PythonLayer can be a loss layer. PythonLayers are derived from the same Layer interface as the rest of the Caffe layers so `setup()`, `forward()`, `backward()`, and `reshape()` are all exposed for any purpose.

See this example PythonLayer for computing the Euclidean loss and its example prototxt. Note the use of `loss_weight` to signify it is a loss.

Evan Shelhamer

--
You received this message because you are subscribed to the Google Groups "Caffe Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/8c905b5b-e2f6-4af4-aed6-bb19b5443a4e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mahdieh Abbasi]

Hi Evan,

Thank you for your answer. In fact, I just need to update the loss function by adding a new regularization term, not totally change the loss function. Like  https://github.com/BVLC/caffe/issues/1855 , I would like to compute the gradient of the new loss function (which has the regularization term) with respect to the last fully connected layer, then pass the computed gradient to net.backward(gradient) to update the parameters with regard the gradient. But I got error. If I tries to change diff of the last layer with my computed gradient, the backward applies my gradient through the network?

Thank you a lot for answering my questions

Sincerely,

 

Mahdieh Abbasi

 

 

--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/ELHfe3KbD0M/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.

To post to this group, send email to caffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/CAEpyZsUazv06zuQiP6Dekb%2BOZQjFRyLR%2BgbNESTu%2B87jM0GO5w%40mail.gmail.com.

[Evan Shelhamer]

If your regularization is summed with the existing loss then define your network with two losses that both have the last fully connected layer as their bottom. Let the first loss be the original desired loss and let the second layer be your regularization implemented as a PythonLayer. Each will compute their bottom diff, these diffs will be summed for the top diff of the fully connected layer, and then backprop will continue as usual.

Evan Shelhamer

[勇翔王]

HI I am trying to implement this paper with caffe : http://ieeexplore.ieee.org/xpl/articleDetails.jsp?reload=true&arnumber=7298880

Which regularized the weight of two classifier while training multitask learning because they want two classifier capture similar feature of fully connected layer.

I am stuck on that I have no idea how to extract the weight of two layer in this multitask learning architecture.  

Would you please give some hint ?  Thank you 

[Amir Abdi]

Can python layers use GPU as well?

[Amal Feriani]

Hello,

I would like to do the same thing as you.

Did you find an answer to your question ?

Thank you 

[Zhaoting Ye]

Hi Amal, 

Do you get any solution to this problem? I am also working on it. 

Thanks