Evaluation Measure for OTE task for Restaurants in English Language

[ABHISHEK SETHI]

HI,

Could you please tell me how to evaluate my predictions for this task. How to calculate F1 score for this.

Sincerely,

Abhishek Sethi

[Dimitris Galanis]

Hello,

Which *task* do you mean? There is no ABSA task in the SemEval of this year (2017).

For ABSA 2016 the evaluation package is available here.

http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools

--> Evaluation-Validation-Submission-Baselines

The same package can also be used for ABSA 2015.

Please also consult our overview papers on how we calculate the evaluation scores.

* http://galanisd.github.io/Papers/2016SemEvalABSATaskOverview.pdf

* http://galanisd.github.io/Papers/2015SemEval_ABSA_overview.pdf

Dimitris.

--
You received this message because you are subscribed to the Google Groups "SemEval-ABSA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to semeval-absa+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ABHISHEK SETHI]

Hi,

I am talking about Task 5 ABSA, SemEval -2016.
I could not understand how to get F1.


For example , the review is "The cheese pizza was good but the service was poor.".  Say the actual targets are "cheese pizza" and "service" but my system returned only "pizza" and "service". Then what will be F1 of my system.

Actually  i want to get the results without converting it to the submission format (XML).

To unsubscribe from this group and stop receiving emails from it, send an email to semeval-absa...@googlegroups.com.

[Dimitris Galanis]

I am talking about Task 5 ABSA, SemEval -2016. 
I could not understand how to get F1. 

For example , the review is "The cheese pizza was good but the service was poor.".  Say the actual targets are "cheese pizza" and "service" but my system returned only "pizza" and "service". Then what will be F1 of my system.

OK now I understand.

 

Actually  i want to get the results without converting it to the submission format (XML).

Then the only solution is to rewrite the evaluation script. For this please consult our overview papers and 

in case that you need more info send me a direct email. 

However, I think that it is *much easier* to convert your output to the ABSA submission format. 

Dimitris

 

To unsubscribe from this group and stop receiving emails from it, send an email to semeval-absa+unsubscribe@googlegroups.com.