Maximum allowed runtime

[Mahmoud Khonji]

Dear PAN,

Any thoughts on how long PAN organizers are welling to wait for a
classifier process to complete before they decide that it isn't worth
waiting for (and terminate it)?

Also, would such extremely slow process be assumed to have:
- 0 AUC/0% c@1,
- listed under "too slow",
- or others (please specify)?

Regards,
Mahmoud
Mahmoud Khonji
Research Assistant 2, College of Engineering(Sharjah Campus)
Khalifa University of Science, Technology & Research (KUSTAR)
P.O. Box: 573, Sharjah, UAE
T : +971-(0)6-5978883
F : +971-(0)6-5611789
E : mkh...@kustar.ac.ae

[Martin Potthast]

Dear Mahmoud,

> Any thoughts on how long PAN organizers are welling to wait for a
> classifier process to complete before they decide that it isn't worth
> waiting for (and terminate it)?

We haven't explicitly specified a time-limit for the processes to
finish, but I would suggest that you keep computations as small as
possible. This is to avoid any difficulties for us to evaluate your
submission in time, since restarting long-running processes is usually
a nuisance.

> Also, would such extremely slow process be assumed to have:
> - 0 AUC/0% c@1,
> - listed under "too slow",
> - or others (please specify)?

We will probably indicate the fact that a software did not complete
execution as something along the "n/a", or "incomplete execution", or
something similar.

Best,
Martin


--
Martin Potthast
Bauhaus-Universität Weimar
www.webis.de --- www.netspeak.org