F1 Score

angel b.

unread,

Apr 22, 2021, 6:23:58 AM4/22/21

to HeuristicLab

Hello

I am trying to use F1 score in Genetic Programming - Symbolic Regression.

I work on the "Best training solution"

The confusion matrix (training) is as follows: TN: 188; FN: 21; FP: 39; TP: 206

The confusion matrix (test) is as follows: TN: 15819; FN: 36; FP: 5481; TP: 191

F1 score (training): 0.86238532110091737

F1 score (test): 0.85151392813887772

I am trying to check the results with the formula:

F1 = 2 * (Precision * Recall) / (Precision + Recall)

or

F1 = TP / (TP + 1/2 * (FP + FN))

Results are different from HL, especially in the test set.

I would like you to tell me what formula F1 score uses.

Thanks

Gabriel Kronberger

unread,

Apr 22, 2021, 6:49:53 AM4/22/21

to noreply-spamdigest via HeuristicLab

Hi Angel,

I suspect this issue is related to the one affecting the Gini index. I'll have a look at it.

Best, Gabriel

--
You received this message because you are subscribed to the Google Groups "HeuristicLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to heuristiclab...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/heuristiclab/4a0cc993-26e7-4c04-b6db-3806b3fb8404n%40googlegroups.com.

Gabriel Kronberger

unread,

Apr 23, 2021, 12:09:30 PM4/23/21

to noreply-spamdigest via HeuristicLab

Hi Angel,

we have checked the code and it indeed calculates the F1 score using the same formula that you specified.

Could you please send an example .hl file that demonstrates the issue to sup...@heuristiclab.com ?

Best, Gabriel

To view this discussion on the web visit https://groups.google.com/d/msgid/heuristiclab/15d2a111-cac1-4066-8d2d-bc36a4a3a9bf%40www.fastmail.com.

angel beade torreiro

unread,

Apr 24, 2021, 11:28:17 AM4/24/21

to heuris...@googlegroups.com

Hi, Gabriel

Attached file for verification

Trying to check the results with the formula:

F1 = 2 * (Precision * Recall) / (Precision + Recall) It implies beta=1

Precision = Positive predictive value

Recall or Sensitivity = True positive rate

Best training solution.Classification Performance Measures.Positive predictive value (test) 1.98%

Best training solution Classification Performance Measures True positive rate (test) 78.85%

Best training solution Classification Performance Measures F1 score (test) 0.735570231

Result of applying the formula 0.038548509

Best training solution.Classification Performance Measures.Positive predictive value (training) 78.31%

Best training solution Classification Performance Measures True positive rate (training) 93.83%

Best training solution Classification Performance Measures F1 score (training) 0.821515892

Result of applying the formula 0.853707415

Best validation solution.Classification Performance Measures.Positive predictive value (test) 2.95%

Best validation solution.Classification Performance Measures.True positive rate (test) 77.53%

Best validation solution.Classification Performance Measures.F1 score (test) 0.841759792

Result of applying the formula 0.056902683

Best validation solution.Classification Performance Measures.Positive predictive value (training) 77.31%

Best validation solution Classification Performance Measures True positive rate (training) 81.06%

Best validation solution Classification Performance Measures F1 score (training) 0.781038375

Result of applying the formula 0.791397849

Best, Angel

Enviado desde Correo para Windows 10

To view this discussion on the web visit https://groups.google.com/d/msgid/heuristiclab/4eafcbe6-4883-4e99-9164-3a3889479498%40www.fastmail.com.