Collection for testing "all IPC domains"

[Christian Muise]

Part of the motivation behind "Collections" as a group of domains was to provide some attempt at having a canonical (or at least reproducible) set of benchmarks that spanned the entire IPC. This is the current description of collection #12:

A selection of STRIPS domains from every IPC so far. The following rules were used to construct the collection:
* Only one copy of each domain exists
* Benchmarks from the satisficing track are selected instead of the optimal track
* The most recent IPC was used if a benchmark appears in more than one year
* Two domains were sampled to bring the number of problems in line with the other benchmarks: blocks has every second problem giving us 45 instances from the 2000 benchmark and logistics has every fifth problem giving us 42 instances from the 2000 benchmark.

The current collection is definitely not up to snuff, and I would welcome any suggestions as to how it can be improved. For those who want to poke around what's there in the collection, you can either use the import functionality on editor.planning.domains ("All-IPC (STRIPS)" is the current name of the collection), or browse the raw API:

- http://api.planning.domains/domains/12

Eventually, the resolution of this ticket should signal the first official version of the collection:

- https://bitbucket.org/planning-tools/api-tools/issue/9/fix-up-the-all-ipc-collection

[Christian Muise]

Seeing as everyone on this list joined /after/ the previous message was sent, I just want to bring it up again. Settlers has been removed, but are there any other glaring omissions? I think someone mentioned that logistics-98 was preferred to logistics-00 because of problem difficulty -- can anyone confirm this?

  Cheers

[Malte Helmert]

On 22.06.2015 09:08, Christian Muise wrote:
> Seeing as everyone on this list joined /after/ the previous message was
> sent, I just want to bring it up again. Settlers has been removed, but
> are there any other glaring omissions? I think someone mentioned that
> logistics-98 was preferred to logistics-00 because of problem difficulty
> -- can anyone confirm this?

Yes, Logistics-2000 is comparatively boring. Logistics involves trucks
that move within the locations of a city and airplanes that move between
cities via airports. (Every city has exactly one airport.)

In Logistics-2000, the truck aspect of the domain is quite trivial:
every city has exactly one truck and exactly two locations (one airport,
one other). Logistics-1998 has varying numbers of trucks per city and
varying city sizes.

Cheers,
Malte