submission format

[ipek baris]

Hi,

I would like to upload the dev dataset results of my model to Codelab. I receive invalid type input error. The format of the results works without any problem when I execute scorer.py locally.

What could it be the problem?

kind regards,

Ipek

--

İpek Barış

[Genevieve Gorrell]

Perhaps Python version?

Genevieve

[Genevieve Gorrell]

As in, does it work locally with the other Python. I think I coded it to work with both 2 and 3, but Codalab uses I can't remember which version. So if you can reproduce the problem locally it might help you more to debug your submission.

Genevieve

[hunnyis...@gmail.com]

You may try to compress your answer.json into a zip file and upload to Codalab. I've encountered the same problem and solved this way.

在 2019年1月8日星期二 UTC+8下午11:42:00，ipek baris写道：

[ipek baris]

Hi all,

I submitted answers as compressed form and the system accepted it as you said. Now I have another problem. The score of evalution for development set is shown as zero in the system. On the other hand, I got these results when executing scorer.py:

Task A (SDQC), English, accuracy: 0.4891745602165088

No truth data found for subtask B, English

No truth data found for subtask A, Danish

No truth data found for subtask B, Danish

No truth data found for subtask A, Russian

No truth data found for subtask B, Russian

Task A (SDQC), accuracy: 0

Task B (veracity), accuracy: 0 RMSE: 0

In the json, I only add taskaenglish field and ignoring the other fields. What am I missing?

Best regards,

Ipek

--
You received this message because you are subscribed to the Google Groups "RumourEval" group.
To unsubscribe from this group and stop receiving emails from it, send an email to rumoureval+...@googlegroups.com.
To post to this group, send email to rumou...@googlegroups.com.
To view this discussion on the web, visit https://groups.google.com/d/msgid/rumoureval/8ac0aeba-3ad8-48a1-889f-40815bf414b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.