New in version 1.2: Command-line arguments parsed with argparse
New in version 1.2: seed.py: flags for specifying the smtp server and port have been added; the logging services have been added; the verbose flag has been added; most of the arguments became optional
New in version 1.2: buildplanner.py: logfile accepts the same placeholders than the build log file; --ini added; use of regular expressions instead of filename pattern expressions; the automated e-mail facility has been added
New in version 1.2: builddomain.py: logfile accepts placeholders; --ini added; use of regular expressions instead of filename pattern expressions; the automated e-mail facility has been added
New in version 1.2: invokeplanner.py: logfile accepts placeholders; --ini added; the verbose flag has been added
Changed in version 1.2: invoke-planner.py: renamed as invokeplanner.py
New in version 1.1: validate.py: the step length is computed now in addition to the value returned by VAL
This is the first of the two important packages developed at the Seventh International Planning Competition. The second one is IPCReport —see IPCReport. A third one, with code which is particular to the IPC 2011 is distributed separately: IPCPrivate described in IPCPrivate. Besides, the IPCTools were developed after the IPC to ease the data analysis and it is described in IPCTools
The IPCData module consists, mainly, of six different python modules: check.py, seed.py, buildplanner.py, builddomain.py, invokeplanner.py and validate.py. All of these programs are discussed in the next sections.
All these modules have been developed with Python 2.x
Warning
Version 1.2 of IPCData uses argparse for parsing the command-line arguments. While they significantly improved the usability of the scripts, it requires either Python 2.7 or Python 3.2
The modules described in this chapter have a number of dependencies with third-party software that have to be installed prior to the installation and usage of IPCData:
Configobj: This package provides a wrapper to read and write INI configuration files
Instructions for downloading and installing this package are given here
Subversion python: This package provides a wrapper to access svn repositories in Python
Instructions for downloading and installing the package are given here
Finally, some modules in this package also use PrettyTable. However, since it consists of a single module (i.e., no __init__ file is given), it is provided within the package IPCData by default.
A step-by-step procedure to install all these packages can be accessed in the first practical case discussed in this manual —see First practical case.
In what follows, it is assumed that the reader has already checked out to her local computer the scripts located at:
svn://svn@pleiades.plg.inf.uc3m.es/ipc2011/data/scripts/pycentral/IPCData
All the modules in this package adhere to a consistent naming of the flags they acknowledge. This section describes this convention.
As a general rule all programs honour, at least, the following three flags:
-h, --help provide a brief description of the main purpose of the script and presents all the available flags -v, --verbose shows additional information to track progress -V, --version shows the current version of the script along with the head svn release that affects it and the building date
Besides, all modules acknowledge a number of directives for configuring the logging services:
-l, --logfile if provided, all output is redirected to the given file. It is common practice for most scripts to recognize a number of placeholders which are just strings starting with $ and can be different in each case [1]. To allow the generation of logfiles with the same name, the current date and time is appended at the end. If a log filename is provided no output is shown in the standard output -L, --level specifies the desired level of the log messages. Legal values are: INFO, WARNING and CRITICAL. Only messages of the same level or above are shown. In all cases, the default value is INFO, i.e., all messages are shown -e, --email accepts an arbitrary number of e-mail recipients to notify upon completion of the current script. If a logfile was specified with --level then it is automatically attached to the body as well.
To configure the automated e-mail facility see seed.py. On the other hand, when applicable, tracks and subtracks are provided with the following directives:
-t, --track track name, e.g., seq or tempo -s, --subtrack subtrack name such as opt or sat
In the same vain, planners and domains can be fully specified with the following arguments:
-D, --domain all domains matching any of the regular expressions given after the directive are accepted by the script -P, --planner likewise, all planners matching any of the regular expressions given here are accepted by the script -X, --exclude-domain all domains matching any of the regular expressions given after the directive are discarded by the script -x, --exclude-planner as with the previous directive, all planners matching any of the regular expressions given here are discarded by the script
The next set of directives refers to other sources of information that are used by most modules in this package:
-b, --bookmark this flag specifies the svn bookmark to be used by the script. If none is given, it tries to retrieve it from the INI configuration file specified with the next argument -i, --ini specify the location of an INI configuration file. If none is provided ~/.ipc.ini is tried by default
The svn bookmark specifies the location of the svn server with the data repository —see Contents of the data repository. The INI configuration file is created with seed.py.
Finally, while most scripts accept an specification of an output directory, validate.py expects to get an input directory. In all cases, the directory is specified with the following directive:
-d, --directory specifies the location of a directory either for reading or writing data. In case the directory is used in write mode, specific subdirectories are created to store data and an exception is raised if they already exist
Most of these parameters accept a single value. However, a number of considerations follow:
If two or more directives have no values, they can be abbreviated in a single argument. For example, -vm specifies both verbose output and test mode in the script invokeplanner.py
Some directives can be specified an arbitrary number of times. For example, two e-mail recipees can be specified with --email albert.einstein@wurttemberg.edu --email kurt.godel@vienna.edu. However, the same command-line arguments can be abbreviated as follows --email albert.einstein@wurttemberg.edu kurt.godel@vienna.edu.
Other directives that accept an arbitrary number of arguments are: --domain, --planner, --exclude-domain and --exclude-planner
Although this script is mainly thought for the organizers of an International Planning Competition, it can be used also by practitioners in their own labs. While it can be easily extended to make additional checks, it fundamentally tries to make sure that the directory where a planner resides looks like a typical directory of the IPC. Bear in mind that this script does not examine the svn repository but a local directory where a particular check out has been performed.
Planners are stored in a directory according to the following convention: planners/<track>-<subtrack>/<track-subtrack>-<name> where <track>, <subtrack> and <name> stand for the track and subtrack of a planner whose name is given in <name>. Besides, every directory of this type shall contain at least the build and plan scripts:
build: it is a script (usually a bash script) that automatically starts the compilation of the planner when invoked. A typical content shall look like:
#!/bin/bash makethough they can be as complex as desired
plan: this script shall receive three parameters: domain, problem and plan solution file and should invoke the planner accordingly.
From the webpage of the IPC 2008, the following example has been taken: the executable for the “Find Plans Quickly” planner is called fpq and accepts the same command-line arguments as the FF planner. Then your plan script may look like:
#!/bin/bash ./fpq -o $1 -f $2 mv $2.soln $3There is only one exception to this rule. If the planner is run in dck mode, then it will be invoked with up to four different arguments, the fourth being the location of the domain control knowledge used at the Learning Track of the Seventh International Planning Competition —see Contents of the data repository
The script check.py accepts the directive --directory to specify a particular directory to examine. A few examples follow.
To check whether the directory where the ArvandHerd planner of the seq-mco track has been stored follows the convention of the IPC:
% ./check.py --directory ~/lab/ipc2011/planners/seq-mco/seq-mco-arvandherd/
----------------------------------------------------------------------
Name : arvandherd
track : sequential
subtrack : multicore
signature : seq-mco-arvandherd
directory : /Users/clinares/lab/ipc2011/planners/seq-mco/seq-mco-arvandherd/
[1 planners found]
where ~/lab/ipc2011/planners/seq-mco/seq-mco-arvandherd/ is the location where this planner was checked out. The name, track and subtrack are automatically extracted from the signature which is read from the given path. Because this planner explicitly says that one planner has been found, it is also ensured that the scripts build and plan where stored at the right locations.
Also, a whole track-subtrack can be examined using the --recursive flag. For example, from the directory structure of the seq-mco track, the following check can be issued:
% ./check.py --directory ~/lab/ipc2011/planners/seq-mco --recursive
----------------------------------------------------------------------
name : acoplan
track : sequential
subtrack : multicore
signature : seq-mco-acoplan
directory : /Users/clinares/lab/ipc2011/planners/seq-mco/seq-mco-acoplan/
----------------------------------------------------------------------
name : arvandherd
track : sequential
subtrack : multicore
signature : seq-mco-arvandherd
directory : /Users/clinares/lab/ipc2011/planners/seq-mco/seq-mco-arvandherd/
----------------------------------------------------------------------
...
----------------------------------------------------------------------
name : yahsp2-mt
track : sequential
subtrack : multicore
signature : seq-mco-yahsp2-mt
directory : /Users/clinares/lab/ipc2011/planners/seq-mco/seq-mco-yahsp2-mt/
[8 planners found]
Finally, the check.py accepts a third flag --questionnaire to show the pddl questionnaire of all the planners that are recognized by the script. If provided, it prints out the contents of that file.
Originally, this script was conceived to write a lot of information to the local filesystem in the form of an INI configuration file to reuse it as much as possible, thus saving a lot of accesses to the SVN repository. In the end, however, it serves only to save a few key fields and also to make sure that access to the SVN repository is feasible.
The following argument is mandatory:
-f, --file specifies the name of the INI configuration file to create. While it is possible to provide arbitrary names to this directive, other scripts will look, by default, for the INI configuration file in ~/.ipc.ini. However, all scripts that handle INI configuration files acknowledge a dedicated directive to override this setting.
Besides, -b or --bookmark is mandatory as well — see Command-line arguments. The following parameters are optional and can be provided just to qualify the current IPC:
-n, --name name of the competition currently under configuration -p, --part every IPC is split in a number of parts. Typical parts are: deterministic, uncertainly, learning, etc. Users other than organizers of an IPC might find this useful to refer to particular sets of experiments by a label -w, --wiki specifies the html address of the wiki used. This can be used in a future to automate uploads to the Wiki site. -c, --cluster specifies the name of the front-end node of the cluster to be used for running all the experiments
The values of these parameters are considered to be free text and are directly copied to the preamble of the INI configuration file
The next group of arguments are required if any of the scripts of the package IPCData is ever expected to use the automated e-mail facility:
-m, --mailuser e-mail account to use when sending automatic e-mail messages -M, --mailpwd password of the e-mail account. This password is not encrypted in the INI configuration file. Besides, be aware that it will be sent over the network when opening the e-mail session -t, --smtp address of the smtp server to use when issuing automatic e-mail messages -o, --port port of the smtp server accepting requests to send e-mails
Because all of this parameters can be easily retrieved by other users, a solution would be to use a public account. Fortunately, Google provides unrestricted access to e-mails of this type. As a matter of fact, the following account was set up for the Seventh International Planning Competition:
mailuser: ipc2011.pleiades@gmail.com mailpwd: ys98NfyMenRs smtp: smtp.gmail.com port: 587
All users of this script are very welcome to use this account or to create others of their own [2].
Also, the script checks the contents of all directories in the SVN server and issues a warning if a track-subtrack is empty —maybe because there are no planner and/or because no domain and problems have been specified.
The script writes the contents of the SVN server in the INI configuration file in different sections according to the following schema: the section [tracks] contains the number and name of the tracks found in the SVN repository. Then, for each one it creates a new section with its name specifying the subtracks and its names. Once again, for every subtrack it creates nested subsections with the number and name of the planners and domains (if any) within that track-subtrack.
This section describes all the command-line arguments accepted by buildplanner.py. For a thorough discussion of the command-line arguments see Command-line arguments.
While planners are stored in the SVN repository, they have to be compiled locally in order to be able to run any experiments. In other words, while the source code (and other important files such as build and plan) have to be stored in the SVN repository, object code such as libraries and executables should be generated locally.
As mentioned in the preceding section (see seed.py) it is possible for this script (and others) to override the SVN bookmark. If a particular INI configuration file is specified with --ini, the svn bookmark is retrieved from the variable svn of the general section. This behaviour can be overriden if a bookmark is explicitly given with --bookmark. If none is provided, then the script takes the svn bookmark specified at the ~/.ipc.ini file by default. If no INI configuration file with that name is found and no bookmark has been specified, the script issues a fatal error and exits.
Since the SVN repository can accept an arbitrary number of planners arranged in an arbitrarily large number of tracks-subtracks, this script accepts regular expressions to specify what planners are required to be compiled and which, among those, shall be excluded. The first expression is given in the mandatory flag --planner while the second is expliclity specified with the optional directive --exclude-planner. Besides, it is mandatory to provide the --track and --subtrack the planners belong to. This is useful to distinguish planners with the same name across different tracks-subtracks.
If the user does not specify any --directory then the planner is checked out and compiled in a subdirectory of the current working directory which is named after the planner. If a directory is given, the planner is checked out and compiled in a directory with the name of the planner under the given directory. Besides, if a log filename is given with --logfile, it is generated in the target directory with the detail specified with --level which is INFO by default. This command-line argument acknowledges the same placeholders than the build log file described below: $track, $subtrack and $planner.
Other than the log file described above, buildplanner.py creates also a build log file. If --logbuild is provided, a verbatim copy of the compilation process is stored in the given file. This is very important to make sure that the compilation went fine and that no errors were issued on the way. For example, some participants to the International Planning Competition can pose special requirements in the form of third-party software. Recording the building process and allowing those users to check the resulting files is mandatory to ensure that their code was properly compiled and linked. On the other hand, if anything ever went wrong a whole trace (with the detail provided by the compilation tools) is reported in these files. The specification of the logfile accepts placeholders which are symbolized with the dollar sign $. In particular, there are three recognized variables: $track, $subtrack and $planner which are substituted with the particular track, subtrack and name of the planner to be built. The default value of this variable is build-$planner.log. The build log file is left at the root where the planner was built.
Finally, it is feasible to use the automated e-mail facility provided with most scripts of this package. If --email is provided, all the recipients specified are sent an automatic e-mail as soon as the script ends. If a log file (specified with --logfile) was given then it is attached to the e-mail as well.
For example, the following command:
$ ./buildplanner.py --track seq --subtrack opt --planner cpt4
--directory ~/seq-opt-cpt4 --logfile 'build.$track-$subtrack-$planner'
--bookmark svn://svn@pleiades.plg.inf.uc3m.es/ipc2011/data
checks out and compiles the planner cpt4 that entered the seq-opt competition [3]. Upon completion the planner is located at ~/seq-opt-cpt4/cpt4 along with a new file named build.seq-opt-cpt4 which contains all the messages issued by the compilation and linking tools chosen by the author. A log file with the whole building process is left at ~/seq-opt-cpt4. Besides, the following info message is shown on the terminal:
[Info: the following planners have been built]
[ [u'cpt4']]
Because all contents of this planner are checked out to the target directory, the scripts build and plan are automatically placed in the right location, the base directory of the planner. Of particular interest here is the script build which is automatically invoked by buildplanner.py to compile the selected planner according to the rules given by the authoritative author. Of course, the local computer compiling the code shall be equipped with all the dependencies specified by the planner at hand. Besides, the build script shall be respectful with the local computer by avoiding sudo lines, for example, or writing/modifying other parts of the local filesystem than its current tree structure.
This section describes all the command-line arguments accepted by builddomain.py. For a thorough discussion of the command-line arguments see Command-line arguments.
This script has been conceived to check out the domain and problem files of a particular track-subtrack and to build the corresponding testsets. It accepts much the same directives than buildplanner.py (see section buildplanner.py) but --logbuild.
First of all, it is possible for this script (and others) to override the SVN bookmark. If a particular svn bookmark is specified with --bookmark it is used for retrieving all domain and problem files as described below. If not, the INI configuration file specified with --ini is used instead. If none is provided, then the script takes the svn bookmark specified at the ~/.ipc.ini file by default. If no INI configuration file with that name is found and no bookmark has been specified, the script issues a fatal error and exits.
The domain and problems to be used shall be referenced by a combination of track and subtrack given with the flags --track and --subtrack respectively. This enables domains with the same name to be used across different tracks which is highly desirable, though sometimes it leads to some redundance when the same problems are used in different tracks [4].
The domains checked out are those specified with the mandatory flag --domain after excluding those, among the ones specified with the previous flag, that meet the regular expression given with the optional flag --exclude-domain. Both flags accept regular expressions.
Once the domains have been fully identified with their name (flags --domain and --exclude-domain) and the track and subtrack they belong to (with --track and --subtrack), the script automatically builds the testsets in a directory named after the domain beneath the target directory given in --directory. If none is given, ./ is used by default. Each testset is shown in a separate directory named as testsetxxx where xxx stands for a number in the interval 000 – 999. The contents of each particular testset are computed as follows:
single: If the domain directory in the SVN repository (see Contents of the data repository) contained only a single file, it is is replicated to all the testsets along with each particular file found in the problems directory.
multi: If the domain directory contained an arbitrary number of files, then the script makes sure that there are as many files in the domain directory as in the problems directory and creates as many testsets as files have been found. In this case, each testset consists of a pair (domain, problem) which are taken after sorting the files in both directories in ascending lexicographical order.
While the domain and problem files can be given any names in the svn repository, the files in each testset are named as domain.pddl and problem.pddl when built in the local computer.
Finally, it is possible to specify a log filename to record the whole process with --logfile which is left at the target directory specified with --directory. To discrimante among different executions of this script, the log filename specification accepts the placeholders $track, $subtrack and $domain which are substituted by their corresponding values. Besides, all the e-mail recipees given with --email are sent an automatic e-mail upon completion. If a log file is specified it is attached to the e-mail as well.
For example, the following command:
$ ./builddomain.py --track seq --subtrack opt --domain 'p'
--directory ~/seq-opt-pdomains --ini ~/tmp/example.ini
--logfile 'build-$domain.$track-$subtrack.log'
--email albert.einstein@wurttemberg.edu kurt.godel@vienna.edu
takes all the domains that appear in the seq-opt track in the SVN repository specified in the INI configuration file at ~/tmp/example.ini whose name matches the regular expression p and creates the testsets in ~/seq-opt-pdomains. When the process has been completed a message like the following is issued:
[Info: the following domains have been built]
[ [u'parcprinter', u'parking', u'pegsol']]
Besides, Albert Einstein and Kurt Godel will be interrupted to know that these domains have been built and an e-mail with the generated logfile will be sent to him. The logfile is named build-p.seq-opt.log.11-11-24.20:10:36
This section describes all the command-line arguments accepted by invokeplanner.py. For a thorough discussion of the command-line arguments see Command-line arguments.
This script lies at the core of IPCData. It uses most of the different python scripts distributed with the package and also various services from the modules buildplanner.py and builddomain.py. Its importance comes from the fact that it fully automates the process of running a particular experiment which involves an arbitrary selection of planners and domains in a computer. It is expected to be, by far, the most used module in this package.
As with other scripts, invokeplanner.py can override the SVN bookmark specified in the ~/.ipc.ini file (if it exists) by providing it at invocation time with --bookmark. If none is given, the SVN bookmark is taken from the IPC INI configuration file specified with --ini. If none exists and --bookmark has not been given, the script exits with an error.
invokeplanner.py is restricted to a selection of planners and domains that meet the track and subtrack provided with the mandatory directives --track and --subtrack. This is, the track and subtrack are applied both to the selection of planners and domains. As a matter of fact, this motivated the use of the flag --test discussed below. Since invokeplanner.py makes use of build-planner.py it also acknowledges the flags --planner and --excludeplanner discussed in buildplanner.py. Similarly, since it silently invokes builddomain.py it also acknowledges the flags --domain and --exclude-domain described in builddomain.py. The last four arguments admit arbitrary regular expressions.
If a target directory is given with --directory then both the planners and domains meeting the previous expressions are checked out and built under the specified directory. If not, ./ is assumed by default. Besides, the target directory specification is used also to store all the results of the experiments.
First, buildplanner.py is invoked and thus, all the selected planners are checked out and built as described in buildplanner.py. Internally, buildplanner.py is invoked with a logfile which is named after the substitutions in build-$planner.log. Besides, it can also generate a log file which records the results of the whole process. This can be specified with --logfile and it recognizes the following placeholders track, subtrack, planner and domain. This logfile can record a lot of information at various levels, such as the current version, release and built date of the script; parameters provided to the script (for the sake of completeness); the particular steps followed for building the planners and domains and the time instants when each planner was invoked with every testset. It finally ends with various pretty ASCII tables (created with PrettyTable.py) that show the overall running time, the overall memory used, the number of problems solved and the number of plan solution files generated by each planner in each domain. Each line shown in this logfile is accompanied by a number of fields that include the current local date and time, the username and hostname where the execution was performed, the name of the script and Python function that issued the message and a string with the message level. The message level can be specified with --level and should be one among INFO, WARNING and CRITICAL. Only messages with a level higher or equal than the one specified are printed out.
After buildplanner.py is over, the planners are located under the target directory specified with --directory.
Next, the testsets for the domains that satisfy the regular expressions given in --domain and --exclude-domain are created. Because it might be easily the case that one might want to run some experiments with some preliminary problems instead of using the official ones, an additional flag --test is acknowledged. If --test (with no arguments) is provided, domains are selected from the directory <t track - subtrack> whereas the planners are checked out from the directory with the same name without the heading t. This avoids the additional burden to create ancilliary directories which simply replicate the planners with strange names when all that is needed is just to run planners with testing domains. This is very useful, for example, to get an overall idea of the difficulty of some domains (especially when they are new) prior to the selection of the definitive problems.
As soon as buildomain.py is over, the testsets are located under the target directory given with --directory.
If --email is used, the specified recipees are sent an e-mail upon completion of the algorithm. Besides, if a log file was specified with --logfile then it is attached to the e-mail as well.
At last, but not least, each planner is allotted a fixed amount of time and space for solving each task. If it does not, it (and all its children) are automatically killed by the script. These limits are set with the directives --timeout (in seconds) and --memory (in Gigabytes). For example, at the Seventh International Planning Competition the bounds used were --timeout 1800 --memory 6. A couple of notes about these limits follow:
- The time is measured as computation time for the planner and all its subprocess/threads. Therefore, the wall-clock time shall be usually expected to be slightly larger than the time specified with the directive --timeout
- The memory allotted to the planner is, from the Python documentation pages: The maximum area of address space which may be taken by the process. In particular, planners creating other subprocesses or threads (take for example planners from the Sequential Satisficing Multi-core Competition or portfolios in any track) are restricted to do not reach this limit in total. This is, taking also into consideration the space taken by all the other suprocess and threads launched by the same planner. In the end, it is the author’s responsibility of each planner to guarantee that the limit is not reached. If so, the planner (and all its children) are automatically killed by invokeplanner.py.
Once all the parameters have been set, invokeplanner.py invokes the corresponding script plan of each selected planner. For a full discussion of the design and purpose of this script see check.py.
During the execution, the script records the values of a number of parameters:
- The total time the planner is taking (taking also into account that of their children) at rather regular intervals of five seconds.
- The total memory the planner is taking (taking also into account that of their children) at rather regular intervals of five seconds.
- The number of processes currently running at rather regular intervals of five seconds.
- The number of threads currently running at rather regular intervals of five seconds.
- The number of solutions found at the end, if any.
- The time elapsed since the beginning when each solution was found.
- The overall running time (in clock-wall time) consumed by the planner.
- The maximum memory consumed by the planner during the whole execution.
- The memory the planner was taking just before voluntarily resuming execution or being killed. Note that this is not necessarily equal to the maximum memory —though this is usually the case. While most planners take memory incrementally, some take and release memory so that they show a memory profile which decays from time to time.
All this information is stored at a separate tree structure rooted at the target directory specified with --directory with the name results/ —for a full discussion of the contents of this directory see The results directory. Beneath this directory, a number of subdirectories with the names of all the planners just built are shown. Under each one, there are as many folders as domains have been built with builddomain.py. Finally, each of these directories contain as many subdirectories as problems have been generated whose name meet the format xxx where xxx is an index in the range 000–999. For example, the results of the execution of the Sequential Optimal planner bjolp when faced with the first problem of the domain floortile are stored in results/bjolp/floortile/000. The following paragraphs describe the contents of each of these directories.
The previous fields are recorded at a logfile, whose name results from a combination of the track, subtrack, planner, domain and problem and looks like _set-opt.bjolp-floortile.000-log —for the case of the first problem of the floortile domain when solved by the Sequential Optimal planner bjolp. For illustration purposes, their contents are shown below:
[Fri, 05-20-11 02:57:55.045234]
Current working directory: /home/plg/seq-opt-floortile/_bjolp.floortile.000
Timeout: 1800 seconds
Memory : 6442450944 bytes
[real-time 5] total_time: 4.92
[real-time 5] total_vsize: 150.93
[real-time 5] num_processes: 4
[real-time 5] num_threads: 4
Number of solutions found: 1
Time/solution : [8]
Overall runtime: 10 seconds
Overall memory : 150.93 Mbytes
Maximum memory : 150.93 Mbytes
[Fri, 05-20-11 02:58:05.093734]
Other information recorded in the form of files is the following (the names of the files actually created in our running example are given for the sake of clarity):
_seq-opt.bjolp-floortile.000-cpu
It shows information on the CPUs used as provided by the /proc Linux filesystem.
An example follows [5]:
[Fri, 05-20-11 02:57:55.045340] * CPUinfo: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 30 model name : Intel(R) Xeon(R) CPU X3470 @ 2.93GHz stepping : 5 cpu MHz : 1200.000 cache size : 8192 KB physical id : 0 siblings : 8 core id : 0 cpu cores : 4 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm ida dts tpr_shadow vnmi flexpriority ept vpid bogomips : 5866.76 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management : ... [Fri, 05-20-11 02:57:55.045701]_seq-opt.bjolp-floortile.000-mem
It shows various fields about the available memory and its usage as provided by the /proc Linux filesystem.
The contents of this file are shown below:
[Fri, 05-20-11 02:57:55.045372] * MEMinfo: MemTotal: 8187144 kB MemFree: 5520668 kB Buffers: 48256 kB Cached: 2287552 kB SwapCached: 0 kB Active: 2018184 kB Inactive: 339268 kB Active(anon): 22096 kB Inactive(anon): 120 kB Active(file): 1996088 kB Inactive(file): 339148 kB Unevictable: 0 kB Mlocked: 0 kB SwapTotal: 43163644 kB SwapFree: 43163644 kB Dirty: 254940 kB Writeback: 0 kB AnonPages: 22012 kB Mapped: 10832 kB Shmem: 236 kB Slab: 108080 kB SReclaimable: 94576 kB SUnreclaim: 13504 kB KernelStack: 1928 kB PageTables: 1828 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 47257216 kB Committed_AS: 79012 kB VmallocTotal: 34359738367 kB VmallocUsed: 299208 kB VmallocChunk: 34359435848 kB HardwareCorrupted: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 7744 kB DirectMap2M: 8372224 kB [Fri, 05-20-11 02:57:55.045725]_set-opt.bjolp-floortile.000-ver
It shows the current version of Linux as provided by the /proc Linux filesystem.
An example (properly intended to fit the page width) follows:
[Fri, 05-20-11 02:57:55.045305] * Version: ['Linux version 2.6.35-23-generic (buildd@allspice) (gcc version 4.4.5 (Ubuntu/Linaro 4.4.4-14ubuntu5) ) #41-Ubuntu SMP Wed Nov 24 11:55:36 UTC 2010\n'] [Fri, 05-20-11 02:57:55.045672]
Besides, invokeplanner.py captures both the standard error and the standard output and stores everything in the files planner.err and planner.log respectively. These files are useful for diagnoses purposes and attention shall be given to them by the authors of planners to ensure that they write useful data.
Finally, other files stored with all of these are the domain.pddl, and problem.pddl picked up and any collection of plan solution files generated by the planner with the syntax plan.soln.n where n stands for a number that increases from 1 in steps of 1; also, the logfile generated by buildplanner.py is copied here for the sake of completeness so that each one of these directories contains information enough to be analyzed on their own, without further dependencies.
To end, a couple of examples follow:
Case 1: In this example, the planners lmtd, yahsp2 and yahsp2-mt from the Temporal Satisficing track are going to be invoked over all problems of the domains crewplanning and sokoban. The results shall be automatically sent both to albert.einstein@wurttemberg.edu and kurt.godel@vienna.edu. A human readable recording of all events shall be directed to a logfile named example-1. In this case, each planner will be given only 15 minutes and 2 Gb of memory:
$ ./invokeplanner.py --track tempo --subtrack sat --planner "lmtd|yahsp.*" --domain "crewplanning|sokoban" --directory ~/case-1 --logfile case-1 --email albert.einstein@wurttemberg.edu kurt.godel@vienna.edu --timeout 900 --memory 2Case 2: The following example just runs the whole Seventh International Planning Competition of the Sequential Satisficing track when using the right SVN bookmark (which is provided as well):
$ ./invokeplanner.py --track seq --subtrack sat --planner ".*" --domain ".*" --directory ~/ipc-2011 --logfile my-ipc-2011 --bookmark svn://svn@pleiades.plg.inf.uc3m.es/ipc2011/data --timeout 1800 --memory 6
Usually, the results/ directory takes a lot of space. Therefore, a good idea might be to copy most of its structure to a separate directory preserving only the domain.pddl, problem.pddl, the plan solution files generated and the log file generated by invokeplanner.py. While the original contents of the results/ directory were named raw/ in the context of the Seventh International Planning Competition, this additional copy was known as val/ since it contains the essential information to be validated thanks to the next script.
A mandatory step not only at the International Planning Competition but also at any experiments run at the home lab is to validate the solutions. To this end, VAL has been used. The latest version at the time of the Seventh International Planning Competition was 4.2.08. However, version 1.1 uses version 4.2.09. A copy of it can be retrieved from http://www.plg.inf.uc3m.es/ipc2011-deterministic/FrontPage/Software
Since the number of plan solution files might be huge, the validation process has been also automated. The script validate.py takes care of it. It receives a single mandatory parameter --directory which specifies the current location of a particular case. If the given directory contains a domain.pddl, problem.pddl and various plan solution files, these are given to VAL and its output is recorded in a separate log file which is named after the track, subtrack, planner, domain and problem at hand which looks like _seq-opt.bjolp-floortile.000-val. If the given directory does not contain these files but there are other subdirectories, the entire tree structure is recursively traversed validating all the cases found along the path.
Besides, if a log file is specified with --logfile then messages issued at the specified --level or above are recorded. In case that one or more e-mail recipees are given with --email, they are all notified upon completion of the script. Note that to enable the automated e-mail facility, it is mandatory to provide an INI configuration file with --ini unless ~/.ipc.ini shall be used.
Henceforth, if all the results of a particular invocation of invokeplanner.py are stored at the directory ~/my-ipc-2011, the following command validates all the solutions generated so far:
$ ./validate.py --directory ~/my-ipc-2011
and leaves a validation log file at each location where it found the essential files to perform the validation. In our running example, the file _seq-opt.bjolp-floortile.000-val contains the following data:
[Thu, 10-06-11 00:11:13.276711]
Solution file: plan.soln
Size : 892
Status : 1
Value : 38
Step length : 25
return code : 0
stderr : []
Number of solution files found: 1
Number of correct solutions found: 1
[Thu, 10-06-11 00:11:13.313025]
The fields shown are self-explanatory. In case of error, the corresponding validation log file shows the appropriate data [6].
Footnotes
| [1] | When using the dollar sign $ in the command line, the shell will always try to expand it to the values of environment variables. This is known as interpolation. To avoid it, strings containing dollar signs shall be embraced between single quotes as in ‘$track-$subtrack.$planner-build’ |
| [2] | Be aware however that reusing the same google e-mail account might make others to access the results of your experiments. In case secrecy is desired, it is highly recommended to set up a different e-mail account and to provide the user and password in the INI configuration file as explained in the text. |
| [3] | Note that for buildplanner.py to run properly there is no need to create the INI configuration file ~/.ipc.ini if --bookmark is provided. |
| [4] | For example, all problems used in the Seventh International Planning Competition in the Sequential Satisficing Multi-core track (seq-mco) were exactly the same ones than those used at the Sequential Satisficing track (seq-sat). Because the scripts developed at the IPC 2011 are always restricted to a particular combination of track and subtrack there is no other way to reuse domain and problem files other than copying them to a separate folder in the SVN repository according to the structure discussed in Contents of the data repository |
| [5] | Incidentally, one of the flags of our processors is ida. I wonder whether Intel already implemented the Iterative-deepening A* in their computers :) |
| [6] | Note that from version 1.1 the validation log contains a new line with the step length of each plan |