Calculating the accuracy of prediction

1,919 views
Skip to first unread message

rsl

unread,
Feb 28, 2016, 11:56:43 AM2/28/16
to Caffe Users
I wanna know how to calculate the accuracy of my prediction? 
the prediction is vector of 1764 float values. I have tried to use sklearn.metrics.accuracy_score, but it keeps giving this error :

y_type not in ["binary", "multiclass", "multilabel-indicator"]
ValueError: continuous-multioutput is not supported


I don't understand how whould I change the predicted values to be compatible to accuracy score function! Do you have any suggestion? 
Thanks

Mohamed Ezz

unread,
Feb 28, 2016, 6:10:00 PM2/28/16
to Caffe Users
There's very little context in your question. 

I'll make assumptions :
1- You're doing classification 
2- You have 2 classes 
3- The float vector has probabilities of the data point being positive 

In that case you should threshold the probability predictions to 0 and 1 (for ex. with threshold=0.5), then make sure the 0/1 vector is of int type. Then sklearn.metrics.accuracy_score should work, it will then compare your 0/1 predictions with 0/1 ground truth labels, and calculate accuracy out of them. I hope that helps.

rsl

unread,
Feb 28, 2016, 6:28:04 PM2/28/16
to Caffe Users
I'm doing regression and prediction of multi-label feature vector which contains the float32. The problem is I don't have classes!

Mohamed Ezz

unread,
Feb 28, 2016, 6:36:48 PM2/28/16
to Caffe Users
So what exactly does the prediction vector have? regression target or multi-labels or what? Please post your complete setup, otherwise it's hard to help.

Btw, sklearn.metrics.accuracy_score is not for regression. Regression is not measured in "accuracy", but have other metrics like RMSE (sklearn.metrics.mean_squared_error)

rsl

unread,
Feb 28, 2016, 6:46:00 PM2/28/16
to Caffe Users
Thanks a lot, I got confused between the regression and classification.
I have two case. In the first one I want to see how accurate my regression loss layer is. And in the second problem I want to know how accurate the predicted vector as an output of fc6 layer is.The predicted target is a feature vector of my image.
Reply all
Reply to author
Forward
0 new messages