[FrontPage] [TitleIndex] [WordIndex

This page summarizes the rules for the deterministic part of IPC-2011. It is meant to be the central place where participants can find everything they need to know to build a good entry for the competition. In particular, you will find information or links about:

Other useful information such as deadlines, updates, and supporting documents/information can be found by following links from the competition homepage. If you still do not find the answer to your question, please use the mailing list to discuss the issue. Issues discussed on the mailing list which are of general importance will be reflected in the contents of this page.


The IPC-2011 deterministic part is divided into three main tracks: sequential, temporal planning and preferences. Each track in turn is divided into two subtracks: satisficing and optimal planning. Besides, a new subtrack has been arranged for this IPC for the first time: a multi-core for the sequential satisficing subtrack. Awards will be given to planners in each subtrack and not aggregated across several subtracks, so each subtrack essentially defines a competition on its own.

The seven subtracks differ in which PDDL features are required of the participating planners, which objective function planners are expected to optimize, and whether suboptimal solutions are considered acceptable.

In each track, there are certain "core features" (e.g. durative actions in the temporal track) that are prerequisites for entering the track, and there are also optional features (e.g. ADL, derived predicates) that may appear in some of the domains. For planners that do not support the optional features, we will offer equivalent domain versions where these features have been compiled away. So it is enough to support the core features to fully participate in a given track. Note, however, that compiled problem formulations are often significantly larger than the original problem formulations, so it may be advantageous to support the optional features directly.

The following links contain a detailed description of each track:

Competition Rules

The deterministic part currently considers three tracks: sequential, temporal and preferences. In all cases, two different subtracks shall be arranged: optimal and satisficing; an additional multi-core subtrack shall be arranged for the sequential part as well.

Potential participants in the preferences track are requested to send an email to agolaya@inf.uc3m.es with the registration form completed. For taking part in the sequential and/or temporal track, please send the registration form by e-mail to carlos.linares@uc3m.es

The evaluation process will comprise two phases: Initially, the competitors will be given a set of representative domain/problem instances to test their planners on their own machines. Then, they will submit a final version of the source code of their planners that will be run by the organizers on the actual competition domains/problems (unknown to the competitors till this time). This way no fine-tuning of the planners will be possible.

PDDL in this year's competition

IPC-2011 will be based on PDDL 3.1, i.e. the same language used in IPC-2008 with no extensions at all. A competing team may participate for any track. However, the competitors are strongly encouraged to participate in every track supported by their planners.

See the PDDL resources for further information

IPC-2011 Domains/Problems

Like previous competitions, we plan to introduce new domains and also re-use several domains from previous competitions. We are actively seeking interesting and challenging new domains for this competition. Therefore, if you know any problem that can potentially be a good competition domain, please send the suggestion by email to one of the organizers (Ángel García Olaya <agolaya AT inf DOT uc3m DOT es>, Carlos Linares López <carlos DOT linares AT uc3m DOT es>). We prefer domains that are inspired by real-world applications, and also can be naturally represented (or closely approximated) by the PDDL fragment of any of the three main tracks of this competition.

Depending on the properties of each domain, a given domain may not be used in all three tracks. The problem set, which is likely to be randomly generated for each domain, can also be different between the two subtracks (satisficing and optimization). In particular, given that it is much harder to find proven optimal solutions than finding a satisficing solution, the problems used in the optimization subtracks are likely to be less complex than the ones used in the satisficing subtracks.

Evaluation schema

As in the IPC 2008, this competition is going to be arranged as a blind evaluation. Thus, instead of having the competitors run their planners on problems provided (incrementally) by the organizers, we as the organizers will run all planners according to our schedule and only report the final results when the competition ends. The actual problem sets used in the competition will not be known to the competitors and will only be made public at the end of the IPC.

We will provide example problems for all tracks so that the participants can test their planners before submitting the final executable. We hope that the examples will cover all possible constraints that appear in the actual testing problems. However, if there is a problem running a participating planner on a given testing problem, we will try our best to work with the participants to figure out the issue. Please check the IPC-2011 homepage for information on when the example problems for all tracks will be available and when we will start running the submitted planners on testing problems.

Evaluation Criteria: We do preserve the same evaluation criteria used in the last IPC. Thus, we emphasize good plan quality and put less emphasis on solving time. If two plans are found within the time and memory limits, then a plan with better quality, as defined by the objective function of each track, will have a better score. Note that this is only applicable for the satisficing subtracks because plans in the optimization track will have equal quality. In the optimization subtracks, only the number of solutions found is relevant for the final planner score. The details on the evaluation criteria are summarized below.

The following rules apply for optimization tracks:

The following rules apply for satisficing tracks (including multi-core sequential satisficing):

Some additional notes:

Supporting Tools: We expect to be able to provide a limited set of supporting tools such as updated planning validation software for PDDL3.1 (VAL) from the Strathclyde Planning Group, PDDL 3.1 syntax checkers and standalone parsers (in a limited set of programming languages).

Miscellaneous Issues

Submission of source code: We require all competitors to make public, through the IPC-2011 website, the source code of the version competing in the competition. This will encourage information sharing and allow independent evaluation and double checking of the competition results by the planning community.

Hardware Platform: The competition will be run in a cluster of 11 nodes. Each node is an INTEL XEON 2.93 GHZ QUAD CORE processor (64 bits) using Linux. Up to 6 GB of RAM memory and 750 GB of hard disk will be available for each planner. Each planner will be run in a single node and no planner will be allowed to run in more than one simultaneously (although in the multicore track it will be allowed to use all the 4 cores of a node). No GPU is available. Memory and time will be externally limited.

Using external memory: External memory search is another trend in the search community that can have a big impact on planner performance, when the amount of main memory is limited. We do not disallow planners that use external memory search in the competition. However, the I/O time needs to be counted against the total allocated time for each run (expected to be 30 minutes at the moment).

2013-10-04 16:00