Compute Platform

[Mark "mak" Roberts]

Hi folks, 

Three of the competition tracks banded together to ask for an Amazon EC2 grant, which will serve as the compute platform.  It may be some time before we hear back.  I'm roughly considering running it similar to how Scott Sanner ran the last probabilistic track.  However, I need to figure out exactly what the scripts look like and set up a testing phase where we can make sure the actual competition will run smoothly.  Over the next week, I'll send more specific details/dates.  

In the meantime, perhaps you can get a head start on playing with EC2?  Scott's former directions may provide a rough starting point, though I expect some steps are stale after 3 years.  I'll be curious to know of any hiccups you may hit.

http://users.cecs.anu.edu.au/~ssanner/IPPC_2011/index.html  (cf. Competition Format Section)

http://users.cecs.anu.edu.au/~ssanner/IPPC_2011/IPPC_2011_Amazon_EC2_Setup.pdf  (be sure to use the FREE Micro Instance until the funding is set...)

Cheers,

mak

[Mark "mak" Roberts]

Hi again!

(the last email for now)

A reminder of the upcoming schedule (all competition deadlines in 23:59 Samoa time -- you are on time anywhere in the world):

• (Mar 7 ICAPS Workshop extensions)
• Mar 12* - Test domains ready (participants expected to test Cloud setup before Mar 23, see below for cloud proposal)
• Mar 30 - Final domains published
• Mar 30 - Beginning of 6 week Learning period
• May 3 - End of 6 week Learning Period 
• May 4-18 - Execution of the final domains w/ and w/out DCK  
• May 18-June 5 - Mak is neck deep in data getting the results done in time for the conference!

* revised due to extended ICAPS workshop deadlines

============ EC2 ==============

I looked into the file structure for EC2 and will be sending out an update by 12 Mar for how this should work.

In the meantime, has anyone had time to play with this?    (I realize that last Friday was ICAPS camera ready, that this Friday is the extended deadline for many workshops.  If you can spare 3 minutes to give me a status update -- even if it's "nope" -- it will help me)

For the compute part, here is the rough outline of steps I've figured out so far (I'll provide more detailed instructions by 12 Mar) :

 -- During the test phase starting 12 Mar -- 

 1) you set up a t1.micro "root" instance using Ubuntu 12.04 LTS (32 bit) with an ESB volume of 30 Gig for your planner; 

 2) you set up this root instance to serve an NSF directory for all other "worker" instances

 3) you run your planner on the root instance plus 5 worker instances to test my scripts

 4) you submit your results and feedback by 23 Mar

 5) you provide me the keys to run your instance for any further testing

 6) i *may* ask you to rerun the test during the testing or learning phase to handle problems that surfaced.

-- for the actual competition beginning 3 May -- 

 7) you update your planner in the instance with the same code you send me for your planner submission

 8) I'll take it, expand it into a m1.medium instance with 100+ instances, and run your planner on the competition problems

Does anyone notice a problem or have a concern?  

mak

[Erez Karpas]

Hi Mak,

I'm still not clear on the interface that we need to implement for the learning part, and on what exactly you will provide for each domain:

1. Will there be a finite number of problems available for learning, an instance generator, or both? 

2. If there is an instance generator, what will be the parameters to be used? 

    If there are some given problems, are they ordered somehow? How were they generated?

    My main concern is that I don't want to train on problems some distribution, and then the test set will be generated using another distribution.

3. Do we need to implement a single script (say ./train) which is given a domain as input and outputs a DCK, or is there some room for us to, say, choose/generate/sort the instances we will train on?

Thanks,

Erez.

--
IPC2014 Learning Track website: www.cs.colostate.edu/~ipc2014
---
You received this message because you are subscribed to the Google Groups "IPC2014-learning" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ipc2014-learni...@googlegroups.com.
To post to this group, send email to ipc2014-...@googlegroups.com.
Visit this group at http://groups.google.com/group/ipc2014-learning.
For more options, visit https://groups.google.com/groups/opt_out.

--

--------------------------------------------------------------
"Adventure is just bad planning."
    Roald Amundsen
    Norwegian Arctic & Antarctic explorer  
    (1872 - 1928)                                     
--------------------------------------------------------------

[Mark "mak" Roberts]

Hi Erez,

Thanks for your note.  

I'm planning to run the competition the same way that the last one was ran, so you'll have an instance generator for each domain.  However, it could cause overfitting if I gave you the parameters (even a range) for the problem instances from the generator.  This is my understanding of the last competition.

The competitors perform the training on their own machines.  So, on 30 Mar, I provide a suite of domain (generators) and each competitor has 6 weeks to create the DCK files.  During that time, we work together to finalize the EC2 instances (if not already done by then).  I will run the planner(s) using the format: "plan domain.pddl problem.pddl solutionFile.txt [dckFile]"

Hopefully this helps?  

(I apologize for the delay.  I've been working on a paper for the ICAPS workshop deadline.)

mak

[Erez Karpas]

Hi Mak,
Thanks for clarifying. Just one more question:
Are we supposed to write a script that intelligently calls the instance generator, or do we call it manually?

Thanks,
Erez

[Mark "mak" Roberts]

Hi Erez,

Each competitor is handling the learning portion, so I don't feel comfortable how this is done.  However, it would seems justified to select the distributions that highlight your approach.  

mak

For more options, visit https://groups.google.com/d/optout.

[Jendrik Seipp]

Am 07.03.2014 12:40, schrieb Mark "mak" Roberts:
> Hi Erez,
>
> Thanks for your note.
>
> I'm planning to run the competition the same way that the last one was
> ran, so you'll have an instance generator for each domain. However,
> it could cause overfitting if I gave you the parameters (even a range)
> for the problem instances from the generator. This is my
> understanding of the last competition.

Hi Mark,

just for your information, I think that the last learning competition
provided both the parameter ranges for the generators and some example
instances when the learning phase started
(http://www.plg.inf.uc3m.es/ipc2011-learning/Rules.html). I am uncertain
whether this is better or worse than your proposed approach, but it
might be very hard to learn DCK if the hardness of the test instances is
not at all clear.

Cheers,
Jendrik

[Mark "mak" Roberts]

Ah, okay.  That makes sense.  Thanks for clarifying this.  

Erez, would this approach satisfy your question(s)?

mak

 

Cheers,
Jendrik

--
IPC2014 Learning Track website: www.cs.colostate.edu/~ipc2014
--- You received this message because you are subscribed to the Google Groups "IPC2014-learning" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ipc2014-learning+unsubscribe@googlegroups.com.
To post to this group, send email to ipc2014-learning@googlegroups.com.

Visit this group at http://groups.google.com/group/ipc2014-learning.

For more options, visit https://groups.google.com/d/optout.

[Erez Karpas]

Yes, that would be great.

If this was specified in some easily machine readable way, that would be even better, since it would mean we could write a program that does instance generation and learning without any human intervention.

Cheers,

Erez.

To unsubscribe from this group and stop receiving emails from it, send an email to ipc2014-learni...@googlegroups.com.
To post to this group, send email to ipc2014-...@googlegroups.com.

Visit this group at http://groups.google.com/group/ipc2014-learning.
For more options, visit https://groups.google.com/d/optout.