Evaluation of Retrieval Performance

[jiangk...@gmail.com]

Could you please upload the code and something related about the retrieval performance part? I would like to test the performance of other IR models on HotpotQA. If not, is replacing the paragraphs in dev set the only way to evaluate other IR models? Hope can get your reply and help which I would really appreciate.

[zhi...@google.com]

It is possible to evaluate IR performance by comparing the retrieved documents against the supporting facts, because it is easy to infer the gold paragraphs from the supporting facts by just taking the titles.

[蒋坤]

OK, thanks for you help~

zhiliny via HotpotQA <hotp...@googlegroups.com> 于2018年12月5日周三 上午11:09写道：

It is possible to evaluate IR performance by comparing the retrieved documents against the supporting facts, because it is easy to infer the gold paragraphs from the supporting facts by just taking the titles.

On Tuesday, December 4, 2018 at 3:34:39 PM UTC-5, jiangk...@gmail.com wrote:

Could you please upload the code and something related about the retrieval performance part? I would like to test the performance of other IR models on HotpotQA. If not, is replacing the paragraphs in dev set the only way to evaluate other IR models? Hope can get your reply and help which I would really appreciate.

--
You received this message because you are subscribed to the Google Groups "HotpotQA" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hotpotqa+u...@googlegroups.com.
To post to this group, send email to hotp...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hotpotqa/5d888e09-d172-44b9-93a8-abdb12d40398%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.