F1 Score

15 views
Skip to first unread message

angel b.

unread,
Apr 22, 2021, 6:23:58 AM4/22/21
to HeuristicLab
Hello
I am trying to use F1 score in Genetic Programming - Symbolic Regression.
I work on the "Best training solution"
The confusion matrix (training) is as follows: TN: 188; FN: 21; FP: 39; TP: 206
The confusion matrix (test) is as follows: TN: 15819; FN: 36; FP: 5481; TP: 191
F1 score (training): 0.86238532110091737
F1 score (test): 0.85151392813887772
I am trying to check the results with the formula:
F1 = 2 * (Precision * Recall) / (Precision + Recall)
or
F1 = TP / (TP + 1/2 * (FP + FN))
Results are different from HL, especially in the test set.
I would like you to tell me what formula F1 score uses.
Thanks

Gabriel Kronberger

unread,
Apr 22, 2021, 6:49:53 AM4/22/21
to noreply-spamdigest via HeuristicLab
Hi Angel,
I suspect this issue is related to the one affecting the Gini index. I'll have a look at it.
Best, Gabriel
--
You received this message because you are subscribed to the Google Groups "HeuristicLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heuristiclab...@googlegroups.com.

Gabriel Kronberger

unread,
Apr 23, 2021, 12:09:30 PM4/23/21
to noreply-spamdigest via HeuristicLab
Hi Angel,

we have checked the code and it indeed calculates the F1 score using the same formula that you specified.
Could you please send an example .hl file that demonstrates the issue to sup...@heuristiclab.com ?

Best, Gabriel

angel beade torreiro

unread,
Apr 24, 2021, 11:28:17 AM4/24/21
to heuris...@googlegroups.com

Hi, Gabriel

 

Attached file for verification

Trying to check the results with the formula:

F1 = 2 * (Precision * Recall) / (Precision + Recall)       It implies beta=1

Precision = Positive predictive value

Recall or Sensitivity = True positive rate

 

 

Best training solution.Classification Performance Measures.Positive predictive value (test) 1.98%

Best training solution Classification Performance Measures True positive rate (test) 78.85%

Best training solution Classification Performance Measures F1 score (test) 0.735570231

Result of applying the formula 0.038548509

 

 

Best training solution.Classification Performance Measures.Positive predictive value (training) 78.31%

Best training solution Classification Performance Measures True positive rate (training) 93.83%

Best training solution Classification Performance Measures F1 score (training) 0.821515892

Result of applying the formula 0.853707415

 

 

Best validation solution.Classification Performance Measures.Positive predictive value (test) 2.95%

Best validation solution.Classification Performance Measures.True positive rate (test) 77.53%

Best validation solution.Classification Performance Measures.F1 score (test) 0.841759792

Result of applying the formula 0.056902683

 

 

Best validation solution.Classification Performance Measures.Positive predictive value (training) 77.31%

Best validation solution Classification Performance Measures True positive rate (training) 81.06%

Best validation solution Classification Performance Measures F1 score (training) 0.781038375

Result of applying the formula 0.791397849

 

Best, Angel

 

Enviado desde Correo para Windows 10

e4 Run 251.hl
Reply all
Reply to author
Forward
0 new messages