class weights for unbalanced data

[Emanuele Dalmasso]

Hi,

I am a new user of dynet, so please excuse me if I write something obvious to the others.

I have a couple of question about a training that I am trying to do.

I am creating a 2 classes classifier with a mlp feedforward net based on the mnist example.

- Given I have unbalanced input data I would like to know if there is a method to weight the classes when doing backpropagation (similar to the class_weight param you can use inside keras wen doing model.fit).

Also I am using the pickneglogsoftmax (postprocessed via sum_batches) for estimating the loss, while in keras I was using the categorical_crossentropy loss function, is it possible to use something more similar to the categorical_crossentropy in dynet?

Thank you very much in advance!

Emanuele.

[Graham Neubig]

(Trying one more time because my first mail bounced)
There's not an explicit method for this, but one way to do it would be to do:

class_weights = ... # Create numpy array of weights per batch member
class_weights = dy.inputTensor(class_weights, batched=True)
log_loss = dy.pickneglogsoftmax_batch(scores, labels)
weighted_log_loss = class_weights * log_loss

Graham

> --
> You received this message because you are subscribed to the Google Groups "DyNet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to dynet-users...@googlegroups.com.
> To post to this group, send email to dynet...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/dynet-users/03d0faad-3e7f-46d5-bfda-ace1d884c06b%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Emanuele Dalmasso]

Thank you!

yes, this is what I was thinking about to do, but I was not sure to be correct, given your answer now I am :-).

I am going to try to implement it now (I am writing in c++ not python).

Thanks,

 Emanuele.