New in version 1.2: Command-line arguments parsed with argparse

New in version 1.1: Welcome IPCTools!

IPCTools

This package was developed after the IPC to ease the data analysis, in particular across tracks/subtracks so that they are available only from version 1.1. As a matter of fact, the modules contained in it serve to create a new (artificial) track with the results of different planners from different tracks with respect to the same set of problems. As soon as data is arranged as explained in The results directory it is then feasible to run the reporting tools described in IPCReport

It consists of two different modules: copy.py and rename.py

Command-line arguments

As a general rule all of these programs have, at least, the following two flags:

-h, --help	provide a brief description of the main purpose of the script and presents all the available flags
-V, --version	shows the current version of the script along with the latest svn release that affected it

The package is located in:

svn://svn@pleiades.plg.inf.uc3m.es/ipc2011/data/scripts/pycentral/IPCTools

All these modules have been developed with Python 2.x

An example of the usage of these package is shown in Third practical case.

copy.py

The first step to perform analysis across tracks consists of creating an artificial new track. For example, to compare the performance of the winners of the sequential satisficing track and the sequential optimal track (lama-2011 and fdss-1 respectively) with regard to the same set of planning tasks, say those in the sequential optimal track, a new track shall be created. In this example it will be known as seqopt-opt. The first contents of this track are created by copying the results of one of the planners to be considered in comparison. copy.py serves explicitly to this goal.

Since copy.py copies files beneath a target directory, it is a good idea to create it explicitly:

$ mkdir ~/tmp/seqopt-opt

This comand will create a new directory to host the new track in the temporary directory. Usually, it is a good idea to start afresh with a new directory since the source directories shall not exist in the destination directory: creating a directory from scracth clearly guarantees this.

This module accepts two directives: --source and --destination. While the latter has to be unique, the former accepts wildcards and can be specified an arbitrary number of times. The module copies all directories matching the source expressions into the destination directory.

In our running example, it is assumed that the results of running lama-2011 with the problems specified in the sequential satisficing track are available in a directory ~/tmp/results/sat-opt/lama-2011. Hence, the following command replicates its contents but this time under a different track/subtrack:

$ copy.py --source ~/tmp/results/sat-opt/lama-2011
          --destination ~/tmp/seqopt-opt

Two important notes:

1. The script automatically realizes that the source directory refers to a planner. Alternatively, it is possible to specify domains or problems.

2. As discussed in The results directory the new results directory shall be created according to the name of the planners/domains/tasks stored therein. Therefore, copy.py renames accordingly all its subdirectories as if it would have been originally created by invokeplanner.py —see invokeplanner.py.

As mentioned above, other directories can be copied to the same destination directory either by providing additional --source directives or just running the script again. For example, executing now (assuming that the results of fdss-1 are stored in the directory specified below):

$ copy.py --source ~/tmp/results/seq-opt/fdss-1
          --destination ~/tmp/seqopt-opt

the new track seqopt-opt will be created with the results of two planners: lama-2011 and fdss-1. Its structure resembles the organization of a legal results directory so that it can be safely examined by the reporting tools —see IPCReport

rename.py

While it might happen that some people might change their mind and want to rename a planner/domain/problem (as it actually happened while running the Seventh International Planning Competition), most people will not use this module directly. It is intended, as its name suggests, to rename a directory which contains a planner, a domain or a particular problem in a results directory, affecting the names of nested files if necessary.

It accepts only two directives: --directory and --name. While --directory has to specify the whole path to the directory to rename (either in absolute or relative formats), the value of the flag --name shall be only the new name of the last folder specified with --directory.

For example, to rename the planner Multiplan.q that took part in the learning track of the Seventh International Planning Competition to PbP2.q, it suffices with:

$ rename.py --directory ~/tmp/results/lrn-sat/Multiplan.q
            --name PbP2.q

This module is automatically invoked by copy.py when copying planners/domains/problems across different tracks so that the resulting tree structure is properly named.