Class weights in binary classification model

[jie....@gmail.com]

We know that we can pass a class weights dictionary in the fit method for imbalanced data in binary classification model. My question is that, when using only 1 node in the output layer with sigmoid activation, can we still apply the class weights during the training? 

model = Sequential()

model.add(Dense(64, activation='tanh'))

model.add(Dense(1, activation='sigmoid')) 

model.compile(

        optimizer=optimizer, 

        loss=loss, 

        metrics=metrics)

model.fit(

        x_train, y_train, 

        epochs=args.e, 

        batch_size=batch_size,

        class_weight={0: 1, 1: 3})

I can also use 2 nodes in the output layer with softmax activation and in this way I can surely use the class weights in the fit method since there are explicitly 2 classes in the output. But I want to use just one node in the output layer since sigmoid is simpler than softmax in my case.

Thanks

[Tomasz Melcer]

On 13.04.2017 12:31, jie....@gmail.com wrote:
> We know that we can pass a class weights dictionary in the fit method
> for imbalanced data in binary classification model. My question is that,
> when using only 1 node in the output layer with sigmoid activation, can
> we still apply the class weights during the training?

Sure, it will work just fine.

The type of output of the network does not actually matter when
specifying weights, because weights are applied to the all output
gradients, not just to a specific output—which, I suspect, is a
misconception you might have.

Besides, if you follow the call stack of the `train` method, you will
see that it invokes an internal `_standardize_user_data` method [1],
which translates class weights into sample weights. In fact, class
weights in Keras is just a convenience API, so that you don't have to
generate sample weights on your own.

[1]
https://github.com/fchollet/keras/blob/2.0.3/keras/engine/training.py#L1277


--
Tomasz Melcer

[Sidharth Kapur]

@Tomasz, are you sure that this will work for binary inputs? I looked through the Keras source code that you linked, and I can't find any code that seems to handle the special case when the number of classes is two (and therefore the prediction is of length 1 instead of length 2).

Thanks!