Some questions about model evaluating

[YY Tong]

Dear organizers,

  I have some questions about the formula to calclulate the f1-score in sub-task A , and i wonder if the result is calculated by the formula below. 

I have used this formula to calculate the f1-score based on your  announced result of other participants' data of precision and recall, and it's not identical with the announced result of f1-score.

  So i want to know your real formula to calculate f1-score. Please tell me the right formula, thank you with my faithful gratitude.

[SemEval2022-iSarcasmEval]

Hi,

The main metric is the F1-score over the sarcastic class (F1-sarcastic). You need the precision and recall to be calculated with respect to the sarcastic class and then using the mentioned equation. The precision and recall reported in the table are the macro-Precision and macro-Recall over the available classes, thus you cannot use them directly to the equation.

You can use the following to calculate the F1-sarcastic:

from sklearn.metrics import f1_score, precision_score, recall_score

f1_sarcastic = f1_score(truths,submitted, average = "binary", pos_label = 1)

OR

p_score_sarcastic = precision_score(truths,submitted, average = "binary", pos_label = 1)
r_score_sarcastic = recall_score(truths,submitted, average = "binary", pos_label = 1)

f1_sarcastic=(2*p_score_sarcastic*r_score_sarcastic)/(p_score_sarcastic+r_score_sarcastic)