Last minute change of the granularity measure

[Martin Potthast]

Dear participants,

please note that the granularity measure has changed (cf. the competition Web page; mind your browser cache).

The changed granularity now computes the base 2 logarithm of the expected number of times a detected plagiarism case will be reported. By taking the logarithm the influence of the granularity on the absolute value of the overall score is demoted. Note, however, that the relative ordering of overall scores will be the same as before.

Best regards,
Martin


--
Martin Potthast
Bauhaus-Universität Weimar
www.webis.de - netspeak.webis.de

If you do things right, people won't be sure you've done anything at all.

[Mike Sanders]

Hello, Martin!

Thanks for the note :-). It will enormously help us - "the participants" of NEXT year competition :-) [I'm just joking!].

By the way - is the deadline the same?
Does the deadline include the 07 day as well? - Say I upload the last set 07? Is it Ok or not?

Looking forward to your reply!
--
Best Regards Yurii

2009/6/6 Martin Potthast <martin....@uni-weimar.de>

[Martin Potthast]

Hi Yurii,

Thanks for the note :-). It will enormously help us - "the participants" of NEXT year competition :-) [I'm just joking!].

;-)

Seriously, it's mostly a quantitative adjustment, and not a qualitative one.
 

By the way - is the deadline the same?
Does the deadline include the 07 day as well? - Say I upload the last set 07? Is it Ok or not?

The deadline is July 7, 2009, 11:59 pm, CET.
But note that we won't be too harsh as this is not a conference with hundreds of submissions.

Best,

[Cristian Grozea]

Dear Mr. Potthast,

So is it June 7th or July 7th?
From the web page:
http://www.uni-weimar.de/medien/webis/research/workshopseries/pan-09/dates.html
> *07.06.2009* Submission deadline for the competition
Best regards,
Cristian

Martin Potthast wrote:
> Hi Yurii,
>
> Thanks for the note :-). It will enormously help us - "the
> participants" of NEXT year competition :-) [I'm just joking!].
>
>
> ;-)
>
> Seriously, it's mostly a quantitative adjustment, and not a
> qualitative one.
>
>
> By the way - is the deadline the same?
> Does the deadline include the 07 day as well? - Say I upload the
> last set 07? Is it Ok or not?
>
>
> The deadline is July 7, 2009, 11:59 pm, CET.
> But note that we won't be too harsh as this is not a conference with
> hundreds of submissions.
>
> Best,
> Martin
>
> --
> Martin Potthast
> Bauhaus-Universität Weimar

> www.webis.de <http://www.webis.de> - netspeak.webis.de
> <http://netspeak.webis.de>

[Martin Potthast]

So is it June 7th or July 7th?

I am sorry, it's my mistake: The deadline is June 7, i.e. tomorrow , as noted on the Web page.

Best,
Martin
 

--
Martin Potthast
Bauhaus-Universität Weimar

www.webis.de - netspeak.webis.de

[Mirko Degli Esposti]

Hi Martin,

you just wrote:

The deadline is July 7, 2009, 11:59 pm, CET. 
But note that we won't be too harsh as this is not a conference with hundreds of submissions.

do you really mean July, or it is  tomorrow June 7 ??

In the first case, we will be even more happy.....we are quite late no the second challenge....

If it is really tomorrow, sunday, as I think,  it could be nice to give time until at least monday 12:00 a.m. or p.m., so 

we don't have to go to the lab tomorrow, sunday, for being ready to submit.....few ph.d. students will be happy for that.....

Regards, mirko

Il giorno 06/giu/09, alle ore 19:32, Martin Potthast ha scritto:

[Cristian Grozea]

Dear Mr. Potthast,

May we suggest/request an extension then, since others would seemingly
benefit from it too? One more reason, yesterday night there was the
deadline for NIPS.
Let's move it two days, it shouldn't change much your checking schedule,
but it would give us a chance not to go to office in weekend.

Best regards,
Cristian Grozea

>> www.webis.de <http://www.webis.de> - netspeak.webis.de
>> <http://netspeak.webis.de>
>>

[Mike Sanders]

Hello, Martin!

Yesterday I was studied the new Granularity formula. If I'm not mistaken (I have a very basic math background) log function discriminates the Granularity resulting value hundreds of times, thus the Granularity influence onto the final result is much less.

I can hardly call it "a quantitative adjustment, and not a qualitative one." (Am I mistaken?).

Actually, I felt that the previous formula's Granularity's influence was too heavy - that is its influence was so big, that received Granularity overrides Recall + Precision effective results. Thus the better result is shown not by the better Plagiarism detection but with a lower Granularity value.

IMHO If I'm getting it right, the new formula much better presents the whole mathematical model.

I'm wondering what provoked you to the formula change? - Was it one of the submitted results analysis?

I would like to share some additional ideas about the Granularity value after the competition I I will manage to get to the conference in person.

The latest formula change will severely hit the teams that work on the Granularity optimization (instead of Recall and Precision)
(if there were such ones, of course). Still, I believe, that were 1-2 teams not more suffered from this change in case when I am correct.

Thank you in advance!
--
Best Regards Yurii

2009/6/6 Martin Potthast <martin....@googlemail.com>

[Martin Potthast]

Hi Yurii,

Yesterday I was studied the new Granularity formula. If I'm not mistaken (I have a very basic math background) log function discriminates the Granularity resulting value hundreds of times, thus the Granularity influence onto the final result is much less.

I can hardly call it "a quantitative adjustment, and not a qualitative one." (Am I mistaken?).

Yes, your are right: the adjustment has both quantitative and a qualitative aspects.
 

Actually, I felt that the previous formula's Granularity's influence was too heavy - that is its influence was so big, that received Granularity overrides Recall + Precision effective results. Thus the better result is shown not by the better Plagiarism detection but with a lower Granularity value.

IMHO If I'm getting it right, the new formula much better presents the whole mathematical model.

I'm wondering what provoked you to the formula change? - Was it one of the submitted results analysis?

Yes, we saw in some preliminary evaluations that granularitie's influence on the overall score was too high. The resulting values were small (i.e., the "psychological" value was "not convincing"), and, more importantly, a reasonable (!) granularity would still disproportionately decrease the overall scores.
 

I would like to share some additional ideas about the Granularity value after the competition I I will manage to get to the conference in person.

That would be great. You may also write about this in your paper, if you like.
 

The latest formula change will severely hit the teams that work on the Granularity optimization (instead of Recall and Precision)
(if there were such ones, of course). Still, I believe, that were 1-2 teams not more suffered from this change in case when I am correct.

That is true, and I would like to apologize for that.
But still, granularity is important both from my understanding of plagiarism detection quality, and from its influence on the overall scores, so their work has not been invalidated.