Re: PAN 2019: Final evaluation data available

54 views
Skip to first unread message

Francisco Rangel

unread,
May 7, 2019, 3:50:07 AM5/7/19
to pan-workshop-series, pan
Dear participants of the Author Profiling shared task on Bots and Gender profiling:

As our colleague Matti told you, the final evaluation dataset is already available.

This means that you have to run your software on this new dataset, as well as the evaluator. 

As you already run your software on the early birds dataset, this procedure should not cause problems. But do not hesitate to contact us otherwise.

Please, do it asap since the final deadline is very close and most of you have not done it yet.

Remember, in the final evaluation only one run is allowed. If you send more runs, we will review only the first one.  

Best regards,

-- 
Francisco M. Rangel Pardo
CTO Autoritas Consulting S.A.
Twitter: @kicorangel

On Fri, 3 May 2019 at 16:57, Matti Wiegmann <matti.w...@uni-weimar.de> wrote:
Dear participants of PAN 2019,

The deadline for the final software submission is approaching: May 11.

The datasets for the final evaluation phase are now available on TIRA:
  - pan19-author-profiling-test-dataset2-2019-04-29
  - pan19-celebrity-profiling-test-dataset2-2019-05-02
  - pan19-cross-domain-authorship-attribution-test-dataset2-2019-05-02

The dataset for Style Change Detection has already been available and
did not change.

If you have any questions, don't hesitate to contact us at p...@webis.de.
Thank you for participating in PAN and good fortune for the last week.

Kind Regards,
Matti Wiegmann



Francisco Rangel

unread,
May 7, 2019, 4:17:38 AM5/7/19
to Marco Polignano, pan-workshop-series, pan
Dear Marco,

You should not try different configurations on the final evaluation set since your system should not be optimised on the test data. In a real-world scenario, you only know training data and have to build and optimise your system acordingly. Here we try to follow the same philosophy. Due to that, we only allow one run. 

Best regards,

On Tue, 7 May 2019 at 10:11, Marco Polignano <marco.p...@uniba.it> wrote:

Dear Francisco M. Rangel Pardo,

what happens if I would try different configurations until the 11th? Can I delete the runs on the final dataset that i do not want to submit leaving just one of them before the deadline?

Best regards,

M.P.


Il 07/05/19 09:49, Francisco Rangel ha scritto:

Paolo Rosso

unread,
May 9, 2019, 1:04:52 AM5/9/19
to JESUSROBERTO LOPEZSANTILLAN, pan, pan-workshop-series, Francisco Rangel
Hola Roberto

Don't worry and take Mexican hour.

Saludos
Paolo

JESUSROBERTO LOPEZSANTILLAN <jrl...@uach.mx> escribió:

> Hi Francisco.
> I’m a participant from Mexico, I was wondering what was the local
> hour of the deadline (May 11, 2019) ?, to know exactly until when I
> have to send send my model. May 11 is Saturday, but maybe I have
> less time since I’m in Mexico.
>
> Thank you very much in advance.
>
> Roberto.
>
>
>
>
>> On May 7, 2019, at 1:49 AM, Francisco Rangel
>> <francisc...@autoritas.es> wrote:
>>
>> Dear participants of the Author Profiling shared task on Bots and
>> Gender profiling:
>>
>> As our colleague Matti told you, the final evaluation dataset is
>> already available.
>>
>> This means that you have to run your software on this new dataset,
>> as well as the evaluator.
>>
>> As you already run your software on the early birds dataset, this
>> procedure should not cause problems. But do not hesitate to contact
>> us otherwise.
>>
>> Please, do it asap since the final deadline is very close and most
>> of you have not done it yet.
>>
>> Remember, in the final evaluation only one run is allowed. If you
>> send more runs, we will review only the first one.
>>
>> Best regards,
>>
>> --
>> Francisco M. Rangel Pardo
>> CTO Autoritas Consulting S.A.
>> http://www.autoritas.net <http://www.autoritas.net/>
>> http://www.kicorangel.com <http://www.kicorangel.com/>
>> Twitter: @kicorangel
>> tlf. +34 656 493 023
>>
>> On Fri, 3 May 2019 at 16:57, Matti Wiegmann
>> <matti.w...@uni-weimar.de
>> <mailto:matti.w...@uni-weimar.de>> wrote:
>> Dear participants of PAN 2019,
>>
>> The deadline for the final software submission is approaching: May 11.
>>
>> The datasets for the final evaluation phase are now available on TIRA:
>> - pan19-author-profiling-test-dataset2-2019-04-29
>> - pan19-celebrity-profiling-test-dataset2-2019-05-02
>> - pan19-cross-domain-authorship-attribution-test-dataset2-2019-05-02
>>
>> The dataset for Style Change Detection has already been available and
>> did not change.
>>
>> If you have any questions, don't hesitate to contact us at
>> p...@webis.de <mailto:p...@webis.de>.

Francisco Rangel

unread,
May 9, 2019, 3:56:46 AM5/9/19
to JESUSROBERTO LOPEZSANTILLAN, pan-workshop-series, pan
Hi Jesus Roberto,

Do not worry for the time, we are not strict with it, TIRA will remain opened.

Best regards,

On Thu, 9 May 2019 at 03:22, JESUSROBERTO LOPEZSANTILLAN <jrl...@uach.mx> wrote:
Hi Francisco.
I’m a participant from Mexico, I was wondering what was the local hour of the deadline (May 11, 2019) ?, to know exactly until when I have to send send my model.  May 11 is Saturday, but maybe I have less time since I’m in Mexico.

Thank you very much in advance.

Roberto.


On May 7, 2019, at 1:49 AM, Francisco Rangel <francisc...@autoritas.es> wrote:

Reply all
Reply to author
Forward
0 new messages