Answers to Questions from the Survey on writing papers

[Ted Pedersen]

Thanks to everyone for responding to the survey about your systems! We
got responses from nearly everyone - if you happened not to respond
yet the survey remains open so please do share that information with
us asap if you haven't already.

We got a number of questions about paper writing on your survey
responses. Here are both the questions and our answers.

---------------------------
Q Should we submit a word or pdf version of the paper?

A: You will submit a pdf version of your paper.
--------------------------
Q: Do we need to include validation F1 scores/results of all the
methods used by us but not submitted as final model?

Not necessarily. What you choose to report about your results is up to
you, however the validation scores can be interesting, particularly if
they were significantly different than what you got as scores on the
test data.
--------------------------
Q: supplementary material is uploaded separately on softconf or with paper?

It does not appear that SemEval is allowing a separate upload of
supplementary material. Given that, you could include such material as
an appendix in the paper (after the references, does not count against
page limits) or you could provide links in your paper to relevant
materials. Remember that SemEval submissions are *not* anonymous, so
links that identify you are not a problem at all.

Related to this, we'd encourage all systems to provide a github site
with any code you'd like to make available, and please include both
references and links to any 3rd party code you used in creating your
system in your paper.
--------------------------
Q: Can we use different headings than given in SemEval guidelines?

The headings provided in the guidelines are suggestions, but you may
choose your own heading names. The following provides a sample outline
for your paper (with heading names) but you are free to deviate from
this:

https://semeval.github.io/system-paper-template.html
--------------------------
Q: How to write the result in the system description? Should we put
avg F1 score in for every phase? bcoz the result table is horizontally
long to be included in the table.

A very important question. As we all know there are a lot of F1
scores. The task organizers paper will include overall results from
the task. You do not need to provide all F1 scores for all phases, you
should focus on those results that you feel are most significant for
your system. In addition to reporting the scores your system achieved,
it is especially interesting and important to do your best to explain
"why" your system got the scores it did. What worked, what didn't, and
specific examples that illustrate this are often very helpful.
--------------------------

We hope these are helpful responses. Please let us know if you have
other questions, or if we need to clarify any of the above. Good luck
in writing your papers, the deadline is a week from today, Tuesday Feb
23. You will submit your papers via the following link :

https://www.softconf.com/acl2021/w13_SemEval/

You are allowed to update your submission before the deadline - so if
you submit a version of your paper on Feb 21 and realize there are
some typos you'd like to fix, you can do that up until Feb 23. The
deadline is Feb 23 anywhere on earth, meaning UTC-12.

Cordially,
Ted
(on behalf of the NLPContributionGraph shared task organizers)
---
Dr. Ted Pedersen
http://www.d.umn.edu/~tpederse