Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

Packaging fast-downguard

23 views
Skip to first unread message

Leopold Palomo-Avellaneda

unread,
Sep 30, 2022, 3:32:09 AM9/30/22
to Fast Downward
Hi,


I would like package fast-downward for Debian (and Ubuntu) to facilitate the
installation of our students.

There are some aspects that are not clear to me and I could like to ask if you
can help. Please, fix it if I'm wrong.

- the build process creates a binary (bin/downward) and to use it, a python
program (fast-downward.py)

- downward (c++) needs a directory translate in the same level to work with?

- the python part in installed in driver??

- which are the minimum files to use with fast-downward?

- And dependencies?
* Python
* GCC or clang
* LP solvers (CPLEX and SoPlex) and OSI
* CMake

cplex is not opensource, soplex neither and not packaged. I have to check OSI
but packaged is only coin-lp. I understood that with only coin-lp should be enough.

Best regards,

Leopold





Florian Pommerening

unread,
Sep 30, 2022, 5:48:39 AM9/30/22
to fast-d...@googlegroups.com
Hi Leopold,

a Debian package would certainly be interesting.

To answer your questions, the Dockerfile of our release probably
contains a lot of useful information (what dependencies to install, how
to build, etc.):
https://github.com/aibasel/downward/blob/main/misc/releases/22.06/Dockerfile.22.06

> - the build process creates a binary (bin/downward) and to use it, a
> python program (fast-downward.py)

There are three components to the planner:
* the search binaries (builds/*/bin/downward)
* the translator Python files (builds/*/bin/translate)
* the driver (fast-downward.py and the directory driver)

I used a * in builds/*/bin as I would suggest to do a release and a
debug build, so the planner can be called with the --debug option.

> - downward (c++) needs a directory translate in the same level to work
> with?

Yes, this is the translator that transforms the PDDL task to SAS^+.

> - the python part in installed in driver??

Yes, this the part that orchestrates the translator and the search
component, runs portfolios, supports aliases, etc.

> - which are the minimum files to use with fast-downward?

You can see that in the Dockerfile I linked above. We use a multi-stage
build where we compile the planner in the first stage and then copy the
compiled files into the second stage. This means that the second stage
has only the files necessary to run the planner. In the Dockerfile you
can see them in the COPY commands in the end.


> - And dependencies?
> * Python
> * GCC or clang
> * LP solvers (CPLEX and SoPlex) and OSI
> * CMake

Only Python (3) is necessary to run the planner (you can see that in the
apt call of the second stage in the Dockerfile). To compile the binary,
there are additional dependencies such as CMake and g++ (clang should
work as well). You can see a list in the apt call of the first stage of
the Dockerfile.

The LP solver is optional and heuristics that rely on it will just not
work without one. We also have some heuristic options that rely on a MIP
solver and this is currently only supported by CPLEX (SoPlex only
handles LPs). If Fast Downward is compiled with an LP/MIP solver, the
solver and the OSI interface for it are also required to run it. This is
what the last two COPY commands in the Dockerfile are about. The ENV
commands set up environment variables necessary so Fast Downward finds
the libraries at runtime.

> cplex is not opensource, soplex neither and not packaged. I have to
> check OSI but packaged is only coin-lp. I understood that with only
> coin-lp should be enough.

If you need an LP solver, CLP (the solver packaged with OSI) should work
but is not tested and we might eventually drop support for it. It also
does not reach the performance of the commercial solvers, so it should
not be used for scientific experiments.

For our releases we have a special permission of ZIB to include SoPlex.
I don't think this agreement would cover your release, so you would have
to get their permission for your use case if you wanted to include SoPlex.



If you want this for a course and assume your students can install *.deb
files, there might be an easier route, though.

Apptainer (formerly Singularity) offers a *.deb file to install it
(https://github.com/apptainer/apptainer/releases). Afterwards, you can
download our latest release as an Apptainer image
(https://www.fast-downward.org/Releases/22.06). The image file can then
be used like a binary.

So for example this should work (untested):

wget
https://github.com/apptainer/apptainer/releases/download/v1.1.0/apptainer_1.1.0_amd64.deb
sudo dpgk -i apptainer_1.1.0_amd64.deb

wget
https://www.fast-downward.org/Releases/22.06?action=AttachFile&do=get&target=fast-downward.sif
chmod a+x fast-downward.sif
./fast-downward.sif --alias lama-first $BENCHMARKS/gripper/prob01.pddl


Cheers
Florian

Malte Helmert

unread,
Sep 30, 2022, 6:35:51 AM9/30/22
to fast-d...@googlegroups.com, Florian Pommerening
On 30.09.22 11:48, Florian Pommerening wrote:
> Hi Leopold,

Hi Leopold,

let me add a few details to Florian's answer.

> > - the python part in installed in driver??
>
> Yes, this the part that orchestrates the translator and the search
> component, runs portfolios, supports aliases, etc.

Well, yes and no. There are two Python parts. One of them is the driver,
the other is the translator.

> > - And dependencies?
> >      * Python
> >      * GCC or clang
> >      * LP solvers (CPLEX and SoPlex) and OSI
> >      * CMake
>
> Only Python (3) is necessary to run the planner (you can see that in the
> apt call of the second stage in the Dockerfile). To compile the binary,
> there are additional dependencies such as CMake and g++ (clang should
> work as well). You can see a list in the apt call of the first stage of
> the Dockerfile.

If you only want to create a "regular" Debian package, not a source
package, the "dependencies" in the Debian sense are only the runtime
dependencies, not the build dependencies.

If you don't enable LP solvers, the only runtime dependency is Python.
(ldd lists some additional standard dependencies like libstdc++, but
perhaps they are part of the base system anyway? If not, check the ldd
output.)

If you enable LP solvers, there are additional runtime dependencies. I
compiled with SoPlex support, and it looks to me like there is *no*
runtime depency on SoPlex, only on OSI-related libraries and on
libraries that OSI's dependencies such as libz. This suggests that
soplex is linked statically. Here is the ldd output:

$ ldd builds/release/bin/downward

linux-vdso.so.1 (0x00007ffe3a1b1000)

libOsiSpx.so.1 => /home/helmert/lib/osi-0.107.9/lib/libOsiSpx.so.1
(0x00007fc2eab1a000)

libOsi.so.1 => /home/helmert/lib/osi-0.107.9/lib/libOsi.so.1
(0x00007fc2eaa8c000)

libCoinUtils.so.3 =>
/home/helmert/lib/osi-0.107.9/lib/libCoinUtils.so.3 (0x00007fc2ea8b0000)

libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fc2ea880000)

libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fc2ea656000)

libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fc2ea56d000)

libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fc2ea54d000)

libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc2ea325000)

/lib64/ld-linux-x86-64.so.2 (0x00007fc2eb018000)

Perhaps someone more knowledgeable about the LP solver setup can comment.

> > cplex is not opensource, soplex neither and not packaged. I have to
> > check OSI but packaged is only coin-lp. I understood that with only
> > coin-lp should be enough.
>
> If you need an LP solver, CLP (the solver packaged with OSI) should work
> but is not tested and we might eventually drop support for it. It also
> does not reach the performance of the commercial solvers, so it should
> not be used for scientific experiments.

Let me reinforce this statement: if there is a published release that
ships with a very poor LP solver, then in our experience people will
start using it in experiments no matter how strongly we say that they
should not. (At least that happened repeatedly with pyperplan and with
the h^m heuristic implementation in Fast Downward.)

From our side this is a problem that we need to address. It would be
nice if people who know what they are doing could still use these (for
example for your course), but if we still had an effective way of
preventing accidental use of these inefficient implementations. Perhaps
we can come up with a special command-line option or similar mechanism
that enables such features.

It would be nice to see performance numbers for CLP so that we can give
more concrete advice on this -- perhaps we can find someone to look into
this.

Best,
Malte

Leopold Palomo-Avellaneda

unread,
Nov 11, 2022, 4:11:51 AM11/11/22
to fast-d...@googlegroups.com, Malte Helmert, Florian Pommerening
Hi fast-downward developers,


I have created a debian package of fast-downward. My main platform is
Debian Bullseye, but I have built it for Debian Bullseye (11) and Ubuntu
Focal 20.04. To install it in:

Debian 11
---------

Follow these instructions:

$ sudo sh -c 'echo "deb [arch=amd64]
http://debrob.upc.edu/debian-robotics bullseye-robotics main" >
/etc/apt/sources.list.d/debian-robotics.list'
$ curl -s https://debrob.upc.edu/debian-robotics/debian-robotics.gpg.asc
| sudo apt-key add
$ sudo apt-get update
$ sudo apt install fast-downward

The sources of the Debian package are located here:

https://salsa.debian.org/lepalom/fast-downward

Ubuntu 20.04
------------

Follow these instructions:

$ sudo add-apt-repository ppa:deb-rob/focal-robotics
$ sudo apt update
$ sudo apt install fast-downward

The ubuntu package is located here:

https://launchpad.net/~deb-rob/+archive/ubuntu/focal-robotics

How is the structure of the package?
------------------------------------

I have built the package without any LP solver. There are created two
binary packages:

* fast-downward

with the binary (C++) executable installed in /usr/bin

* python3-fast-downward

with the python3 code installed in
/usr/lib/python3/dist-packages/downward

When you install fast-downward, by dependencies, it installs
python3-fast-downward.

Patches
-------

I have had to patch the sources in some files. I really don't understand
some design paths or parts of the code. The patches can be viewed here:

https://salsa.debian.org/lepalom/fast-downward/-/tree/master/debian/patches

1) fast-downward.py, as is installed in /usr/bin the Debian Policy obey
to rename it without the .py extension.

2) 0001-Rename-driver-by-downward.patch

The python stuff of fast-downward is installed in:

/usr/lib/python3/dist-packages/downward/

fast-downward is patched to search in downward directory.

3) 0002-Fix-install-paths.patch

I have added two function to return where is installed downward and
translate.py

4) 0003-Fix-portfolio-files-path.patch and
0004-Adding-missing-import-glob.patch

Modified function to obtain the list of the aliases. I have been forced
to do this patch because when the package is installed, it creates a
__pychache__ directory. The original code read the contents (all) and
with an entry with different name that .py it crashed.


Tests
-----

I have run some test, for example:

$ fast-downward --alias seq-sat-lama-2011
misc/tests/benchmarks/gripper/prob01.pddl


And I have not noticed any problem. However, some more test should be
necessary. Remember, there's not LP solver option.

Also, the Debian scripts, at least in Debian, build a
fast-downward-dbgsym package with the debugging symbols, so it is
possible to debug the package built.


Final remarks
-------------

The Debian Policy demands to write a short description and long
description of every binary package. In this case, it can be almost the
same with one of the Python part than the other the C++ binary exec.

But, I'm really confused because I have not been able to describe in a
line what is fast-downward and in some lines a better long description.
Could you help me please?

Also, I can build it with the coinor interface activated, but I don't
know if it could be interesting for you or not. If you agree, I can try
to push it to the Debian Archive as official package, but I would like
to be sure that you agree the changes I have done, and more tests would
be necessary.

Cheers,


Leopold

--
--
Leopold Palomo-Avellaneda <leopold...@upc.edu>
Institut d'Organització i Control de Sistemes Industrials -IOC-
Universitat Politècnica de Catalunya -UPC-

Institute of Industrial and Control Engineering
Technical University of Catalonia
Avda. Diagonal 647, pl. 11
08028 BARCELONA (Spain)

Tel. +34-934016655 (office)
Tel. +34-934017163 (lab)
Fax. +34-934016605

Malte Helmert

unread,
Nov 11, 2022, 4:17:18 AM11/11/22
to Leopold Palomo-Avellaneda, fast-d...@googlegroups.com, Florian Pommerening
Hi Leopold,

cool, that's great!

I had a look at the patches to see if we can integrate something to make
this easier in the future, but I think for now we won't. We cannot
hard-code the absolute paths to the different executables in the same
way because they won't always be in these places, and I think for now
we'll leave the name for the driver packages as "driver" rather than
"downward". But the separation between driver and actual planning code
might be revisited at some point in the future, and then perhaps we can
revisit this.

Best,
Malte

Leopold Palomo-Avellaneda

unread,
Nov 11, 2022, 4:30:40 AM11/11/22
to Malte Helmert, fast-d...@googlegroups.com, Florian Pommerening
Hi Malte,

answering inside:


El 11/11/22 a les 10:17, Malte Helmert ha escrit:
> Hi Leopold,
>
> cool, that's great!
>
> I had a look at the patches to see if we can integrate something to make this
> easier in the future, but I think for now we won't. We cannot hard-code the
> absolute paths to the different executables in the same way because they won't
> always be in these places,

yes, it's true. I don't like it neither, but it is a provisional step. Let me
see how I can do it in CMake (the right tool for that).


and I think for now we'll leave the name for the
> driver packages as "driver" rather than "downward". But the separation between
> driver and actual planning code might be revisited at some point in the future,
> and then perhaps we can revisit this.

Ok, in any case, please, write me some lines about what is fast-downward or how
to describe it. And, I would like that someone check it. I can prepare a package
for Ubuntu 22.04 too.

Best regards,

Leopold
Reply all
Reply to author
Forward
0 new messages