running software

[Ramy Baly]

Hello,

Do we necessarily need to run a .sh file? or can we run a python script directly? i.e.,

 python3 predict.py -i inputDataset -o $outputDir

Do we also need to run the evaluator? or this is done by the organizers?

Thanks,

-Ramy

[Fredrik Johansson]

You can run the Python script directly. To the best of my knowledge you should run the evaluator yourself,  the results will however be hidden from you (when applied on the test set) and only be available to the organizers.

Skickat från min iPhone

--
--
You received this message because you are subscribed to the Google Group "PAN".
Visit this group at http://groups.google.com/group/pan-workshop-series
To unsubscribe send email to pan-workshop-se...@googlegroups.com.
---
You received this message because you are subscribed to the Google Groups "PAN Workshop Series on Digital Text Forensics" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pan-workshop-se...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Johannes Kiesel]

Hi all,

Fredrik is correct on both (Thanks!).

However, you can always ask me to check if your run (on the test set)
was successful. I've also been periodically checking the runs and will
continue to do so.

Regards,
Johannes

On 23.01.19 18:54, Fredrik Johansson wrote:
> You can run the Python script directly. To the best of my knowledge you
> should run the evaluator yourself,  the results will however be hidden
> from you (when applied on the test set) and only be available to the
> organizers.
>
> Skickat från min iPhone
>
> 23 jan. 2019 kl. 18:28 skrev Ramy Baly <ramy...@gmail.com

> <mailto:ramy...@gmail.com>>:

>
>> Hello,
>>
>> Do we necessarily need to run a .sh file? or can we run a python
>> script directly? i.e.,
>>
>>  python3 predict.py -i inputDataset -o $outputDir
>>
>>
>> Do we also need to run the evaluator? or this is done by the organizers?
>>
>> Thanks,
>> -Ramy
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Group "PAN".
>> Visit this group at http://groups.google.com/group/pan-workshop-series
>> To unsubscribe send email to
>> pan-workshop-se...@googlegroups.com

>> <mailto:pan-workshop-se...@googlegroups.com>.

>> ---
>> You received this message because you are subscribed to the Google
>> Groups "PAN Workshop Series on Digital Text Forensics" group.
>> To unsubscribe from this group and stop receiving emails from it, send
>> an email to pan-workshop-se...@googlegroups.com

>> <mailto:pan-workshop-se...@googlegroups.com>.

>> For more options, visit https://groups.google.com/d/optout.
>

> --
> --
> You received this message because you are subscribed to the Google Group
> "PAN".
> Visit this group at http://groups.google.com/group/pan-workshop-series
> To unsubscribe send email to
> pan-workshop-se...@googlegroups.com.
> ---
> You received this message because you are subscribed to the Google
> Groups "PAN Workshop Series on Digital Text Forensics" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pan-workshop-se...@googlegroups.com

> <mailto:pan-workshop-se...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.

--
--
Johannes Kiesel

Bauhaus-Universität Weimar
Bauhausstr. 11, Room 109
99423 Weimar, Germany

Phone: +49 (0)3643 - 58 3720

[Ramy Baly]

Thank you all. I am currently running the software on the training-by-article dataset.

is there a limit on the number of times I can run the software? and how should I specify which run(s) should be considered at the end?

Thanks again,

-Ramy

[Johannes Kiesel]

Hi Ramy,

Thanks for your submission! (glad to have you all working on the task!)

There is no limit. By default we will consider the last two on the
by-publisher test set and the last two on the by-article test set. But
if you want us to consider other runs, just shoot a mail to the
organizers mailing list and specify the runs.

Do not hesitate to ask in case of problems. Just note that I might not
be too responsive in a few hours (it is 20:30 here at the moment).

Regards,
Johannes

[Ramy Baly]

Hi Johannes,

I just finished the run (on the train-byarticle data). No stderr was displayed, but also no file was output. I specified the output directory to be in my home directory.. /home/username/system/

I noticed in TIRA guide that the output directory should look like:  /tmp/username/2018-11-22-10-11-19/output

Is that kind of a restriction for where predictions should be printed? or the problem could be something else.

Apologies for the spam.

On Wednesday, January 23, 2019 at 12:28:25 PM UTC-5, Ramy Baly wrote:

[Vertika]

I have few questions which might sound trivial:

1. Can you elaborate about submissions using web interface. I have successfully run my software via ssh on available train dataset. 

My code is in : /home/username/submissions folder, So should I also register submissions in Working Directory ( for Web Interface ).

Also if I give this $inputDataset, will it automatically resolve to the dataset I have chosen from the menu, Input dataset ( Web Interface).

2. What will variable $outputDir resolve to ? To Register for Test data what should I give in $outputDir. 

3. Can I run and test my registered command on Web Interface for Train data ?

4. The page says: We can have 3 runs on test data and leaving early bird, so we just have 2 till deadline. You mentioned in one of the replies 

"There is no limit. By default we will consider the last two on the by-publisher test set and the last two on the by-article test set. But if you want us to consider other runs, just shoot a mail to the organizers mailing list and specify the runs. ", So now on test also we can ran as many times as we want just that last 2 will be considered, right ?

 You will be able to designate up to three test-data runs for this competition: one submitted before 13 Dec ("Early Bird") and two more submitted before 23 Jan.You will be able to designate up to three test-data runs for this competition: one submitted before 13 Dec ("Early Bird") and two more submitted before 23 Jan.You will be able to designate up to three test-data runs for this competition: one submitted before 13 Dec ("Early Bird") and two more submitted before 23 Jan. 

[Lun Yeh]

As far as I understand,

1. Yes, you should register your working directory. I put SemEval2019 as my working directory as my code resides in /home/username/SemEval2019.

If you choose the dataset you want to run your software on in the web interface, the  $inputDataset  should resolve to the right path for you.

2. On this page, there is the actual command that is run: https://pan.webis.de/semeval19/semeval19-web/tira.html

3. Through the web interface, you only have access to these variables: $inputDataset, $inputRun, $outputDir, $dataServer, and $token

4. You can run as many times as you want on the test set but you won't be able to see the results. And only the last two runs would be considered.

I think you can just test things out through the interface, as failed runs wouldn't matter.

I hope this helps!

--

--
You received this message because you are subscribed to the Google Group "PAN".
Visit this group at http://groups.google.com/group/pan-workshop-series
To unsubscribe send email to pan-workshop-se...@googlegroups.com.
---
You received this message because you are subscribed to the Google Groups "PAN Workshop Series on Digital Text Forensics" group.

To unsubscribe from this group and stop receiving emails from it, send an email to pan-workshop-se...@googlegroups.com.

[Johannes]

Thanks Lun!

Just to add:

2. you do not need to worry what $outputDir resolves to. It is just a
temporary directory your software should write the results to and from
which the results are then gathered.
3. Yes.
4. Sorry for the confusion, but you got it right: you can try as often
as you want; the last two are considered.

Thanks all for being so helpful!
Johannes

Am 24.01.19 um 07:25 schrieb Lun Yeh:

> As far as I understand,
> 1. Yes, you should register your working directory. I put SemEval2019 as
> my working directory as my code resides in /home/username/SemEval2019.
> If you choose the dataset you want to run your software on in the web
> interface, the $inputDataset  should resolve to the right path for you.
> 2. On this page, there is the actual command that is run:
> https://pan.webis.de/semeval19/semeval19-web/tira.html
> 3. Through the web interface, you only have access to these variables:

> |$inputDataset|, |$inputRun|, |$outputDir|, |$dataServer|, and |$token|

> 4. You can run as many times as you want on the test set but you won't
> be able to see the results. And only the last two runs would be considered.
>
> I think you can just test things out through the interface, as failed
> runs wouldn't matter.
> I hope this helps!
>
>
>
>
> On Thu, Jan 24, 2019 at 6:31 AM Vertika
> <vertika.sri...@gmail.com

> <mailto:vertika.sri...@gmail.com>> wrote:
>
> I have few questions which might sound trivial:
>
> 1. Can you elaborate about submissions using web interface. I have
> successfully run my software via ssh on available train dataset.
> My code is in : /home/username/submissions folder, So should I also

> register *submissions *in *Working Directory ( for Web Interface ).*

> Also if I give this $inputDataset, will it automatically resolve to
> the dataset I have chosen from the menu, Input dataset ( Web Interface).
>
>

> 2. What will variable $outputDirresolve to? To Register for Test

> <mailto:pan-workshop-series%2Bunsu...@googlegroups.com>.

> ---
> You received this message because you are subscribed to the Google
> Groups "PAN Workshop Series on Digital Text Forensics" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to pan-workshop-se...@googlegroups.com

> <mailto:pan-workshop-se...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.
>

> --
> --
> You received this message because you are subscribed to the Google Group
> "PAN".
> Visit this group at http://groups.google.com/group/pan-workshop-series
> To unsubscribe send email to
> pan-workshop-se...@googlegroups.com.
> ---
> You received this message because you are subscribed to the Google
> Groups "PAN Workshop Series on Digital Text Forensics" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pan-workshop-se...@googlegroups.com

> <mailto:pan-workshop-se...@googlegroups.com>.

> For more options, visit https://groups.google.com/d/optout.