Calculating multiclass classification bias for softmax and sigmoid layers

20 views
Skip to first unread message

Pejvak Moghimi

unread,
Sep 28, 2020, 7:02:43 AM9/28/20
to TensorFlow End Users - GETTING STARTED, TUTORIALS & HOW-TO'S
Hi all,

Does anyone confidently know what the exact system of equations would be for calculating what the bias would be for the softmax layer with categorical_crossentropy, in a 9 class problem. Furthermore, would it be the same system of equations if one would use a Sigmoid output layer with binary_crossentropy?

I cannot find a source that lays the mathematical foundations of calculating bias accurately.

Cheers,
Pej

Pejvak Moghimi

unread,
Sep 28, 2020, 7:11:01 AM9/28/20
to TensorFlow End Users - GETTING STARTED, TUTORIALS & HOW-TO'S
This notebook on tensorflow demonstrates how to calculate such bias for a binary classification problem, but, not for a multiclass classification problem. Just to note, when I asked for calculating bias for sigmoid-layer with binary_crossentropy, I meant for exact same problem with 9 classes. 
Reply all
Reply to author
Forward
0 new messages