Bug#623485: python-apt: please add bindings pkgOrderList
Pietro Abate
Version: 0.7.100.3+b1
Severity: wishlist
Hello,
This is a request to add a new binding to the class pkgOrderList of apt.
The idea is to allow to compute and set the pkgOrderList independently
and reason about it.
At the moment the only way to compute (via apt-get) the installation
plan is to execute it and print a debug trace of it.
For the mancoosi project we would like to compute the plan before
executing it and to simulate the installation in order to detect
possible problems in the installation scripts.
I'm aware that this binding will export a read-only object and that the
installation plan will be recomputed by apt-get. This will be sufficient
for our goals.
Provided a bit of guidance (I'm not very familiar with the apt python
bindings), we'll be happy to provide a patch.
look forward for your feedback.
pietro
-- System Information:
Debian Release: wheezy/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.32-5-xen-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.utf8, LC_CTYPE=en_US.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages python-apt depends on:
ii apt [libapt-pkg4.10] 0.8.14.1 Advanced front-end for dpkg
ii apt-utils [libapt-inst1.2] 0.8.14.1 APT utility programs
ii libc6 2.11.2-13 Embedded GNU C Library: Shared lib
ii libgcc1 1:4.6.0-3 GCC support library
ii libstdc++6 4.6.0-3 The GNU Standard C++ Library v3
ii python 2.6.6-14 interactive high-level object-orie
ii python-apt-common 0.7.100.3 Python interface to libapt-pkg (lo
ii python2.6 2.6.6-8+b1 An interactive high-level object-o
ii python2.7 2.7.1-6 An interactive high-level object-o
Versions of packages python-apt recommends:
ii iso-codes 3.25-1 ISO language, territory, currency,
ii lsb-release 3.2-27 Linux Standard Base version report
ii python2.6 2.6.6-8+b1 An interactive high-level object-o
Versions of packages python-apt suggests:
pn python-apt-dbg <none> (no description available)
pn python-apt-doc <none> (no description available)
ii python-gtk2 2.17.0-4+b2 Python bindings for the GTK+ widge
pn python-vte <none> (no description available)
-- no debconf information
--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Julian Andres Klode
> Package: python-apt
> Version: 0.7.100.3+b1
> Severity: wishlist
>
>
> Hello,
>
> This is a request to add a new binding to the class pkgOrderList of apt.
> The idea is to allow to compute and set the pkgOrderList independently
> and reason about it.
>
> At the moment the only way to compute (via apt-get) the installation
> plan is to execute it and print a debug trace of it.
>
> For the mancoosi project we would like to compute the plan before
> executing it and to simulate the installation in order to detect
> possible problems in the installation scripts.
>
> I'm aware that this binding will export a read-only object and that the
> installation plan will be recomputed by apt-get. This will be sufficient
> for our goals.
then you order it, and then you read it again. So, shall I bind just the
pkgOrderList class or do you want anything related as well?
> Provided a bit of guidance (I'm not very familiar with the apt python
> bindings), we'll be happy to provide a patch.
I'm going to write it next week, the 0.8.0~exp4 release next
Tuesday/Wednesday would then include it.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
Pietro Abate
On Thu, Apr 21, 2011 at 03:53:39PM +0200, Julian Andres Klode wrote:
> On Mi, 2011-04-20 at 19:06 +0200, Pietro Abate wrote:
> > This is a request to add a new binding to the class pkgOrderList of apt.
> > The idea is to allow to compute and set the pkgOrderList independently
> > and reason about it.
> >
> > At the moment the only way to compute (via apt-get) the installation
> > plan is to execute it and print a debug trace of it.
> >
> > For the mancoosi project we would like to compute the plan before
> > executing it and to simulate the installation in order to detect
> > possible problems in the installation scripts.
> >
> > I'm aware that this binding will export a read-only object and that the
> > installation plan will be recomputed by apt-get. This will be sufficient
> > for our goals.
> As far as I can tell, the object would be read-write. You fill the list,
> then you order it, and then you read it again. So, shall I bind just the
> pkgOrderList class or do you want anything related as well?
If I read the code correctly, you initialize the pkgOrderList with the
cache (that contains the packages marked to be installed/removed) and
then you read back the list of ordered packages. In this sense the
object is of course read/write.
My remark was related how the pkgOrderList class is used in the module
packageManager where the orderedlist is protected. Without modifying
the API of the packageManager it is not possible at the moment to pass
to the installer the installation plan. This is also because the
computation of the installation plan is intertwined with the
real installation itself (for performances reasons I presume).
What we need is a way to build the installation plan from the cache
(initialized with the packages that are going to be installed/removed).
There are a bunch of protected functions in packageManager that use the
orderedList, but as far as I can see they are all protected...
Once the pkgOrderList class will be available I think I'll need to
reimplement in my application the logic of few functions in
packageManager to (re)-build the installation plan (aka the order in
which dpkg is called to pre-configure / unpack / configure / install /
remove / *-hooks / ext ).
From my understanding of the apt code this is the easiest way to attain
this goal... and it's also a lot of work that I prefer to do in python
the to ack the c++ code base of apt-get.
> > Provided a bit of guidance (I'm not very familiar with the apt python
> > bindings), we'll be happy to provide a patch.
> I'm going to write it next week, the 0.8.0~exp4 release next
> Tuesday/Wednesday would then include it.
thank you ! this is fantastic !
regards.
pietro
--
----
http://en.wikipedia.org/wiki/Posting_style
Julian Andres Klode
> hello
>
> On Thu, Apr 21, 2011 at 03:53:39PM +0200, Julian Andres Klode wrote:
> > On Mi, 2011-04-20 at 19:06 +0200, Pietro Abate wrote:
> > > This is a request to add a new binding to the class pkgOrderList of apt.
> > > The idea is to allow to compute and set the pkgOrderList independently
> > > and reason about it.
> > >
> > > At the moment the only way to compute (via apt-get) the installation
> > > plan is to execute it and print a debug trace of it.
> > >
> > > For the mancoosi project we would like to compute the plan before
> > > executing it and to simulate the installation in order to detect
> > > possible problems in the installation scripts.
> > >
> > > I'm aware that this binding will export a read-only object and that the
> > > installation plan will be recomputed by apt-get. This will be sufficient
> > > for our goals.
> > As far as I can tell, the object would be read-write. You fill the list,
> > then you order it, and then you read it again. So, shall I bind just the
> > pkgOrderList class or do you want anything related as well?
>
> If I read the code correctly, you initialize the pkgOrderList with the
> cache (that contains the packages marked to be installed/removed) and
> then you read back the list of ordered packages. In this sense the
> object is of course read/write.
>
> My remark was related how the pkgOrderList class is used in the module
> packageManager where the orderedlist is protected. Without modifying
> the API of the packageManager it is not possible at the moment to pass
> to the installer the installation plan. This is also because the
> computation of the installation plan is intertwined with the
> real installation itself (for performances reasons I presume).
>
> What we need is a way to build the installation plan from the cache
> (initialized with the packages that are going to be installed/removed).
> There are a bunch of protected functions in packageManager that use the
> orderedList, but as far as I can see they are all protected...
>
> Once the pkgOrderList class will be available I think I'll need to
> reimplement in my application the logic of few functions in
> packageManager to (re)-build the installation plan (aka the order in
> which dpkg is called to pre-configure / unpack / configure / install /
> remove / *-hooks / ext ).
But OrderList being available in the Python bindings only brings you the
ordering itself, no integration whatsoever with PackageManager. All the
interesting stuff in PackageManager is protected and thus not bound.
And binding OrderList is not fun, it's severely under-documented. That's
a pain to bind: Because the bindings are fully documented, I need to
play detective and find out what functions and values are supposed to
mean...
--
Julian Andres Klode - Debian Developer, Ubuntu Member
--
Julian Andres Klode
>
> On Thu, Apr 21, 2011 at 03:53:39PM +0200, Julian Andres Klode wrote:
> > On Mi, 2011-04-20 at 19:06 +0200, Pietro Abate wrote:
> > > This is a request to add a new binding to the class pkgOrderList of apt.
> > > The idea is to allow to compute and set the pkgOrderList independently
> > > and reason about it.
> > > Provided a bit of guidance (I'm not very familiar with the apt python
> > > bindings), we'll be happy to provide a patch.
> > I'm going to write it next week, the 0.8.0~exp4 release next
> > Tuesday/Wednesday would then include it.
>
> thank you ! this is fantastic !
Attaching a diff for review. Also see
http://apt.alioth.debian.org/python-apt-doc/library/apt_pkg.html#apt_pkg.OrderList
Note that this only adds a new OrderList class, it does not provide any
integration with PackageManager. If you wish to have integration, please
tell me how you'd like it to be.
Pietro Abate
On Wed, Apr 27, 2011 at 01:07:49PM +0200, Julian Andres Klode wrote:
> On Do, 2011-04-21 at 16:57 +0200, Pietro Abate wrote:
> > On Thu, Apr 21, 2011 at 03:53:39PM +0200, Julian Andres Klode wrote:
> > > On Mi, 2011-04-20 at 19:06 +0200, Pietro Abate wrote:
> > > > This is a request to add a new binding to the class pkgOrderList of apt.
> > > > The idea is to allow to compute and set the pkgOrderList independently
> > > > and reason about it.
> Attaching a diff for review. Also see
> http://apt.alioth.debian.org/python-apt-doc/library/apt_pkg.html#apt_pkg.OrderList
>
> Note that this only adds a new OrderList class, it does not provide any
> integration with PackageManager. If you wish to have integration, please
> tell me how you'd like it to be.
This is great. I had a look at the code and doc and it seems good so
far. I re-read the apt-pkg code regarding the packagemanager class and
maybe I've a suggestion on a possible integration with the work you have
done.
the packagemanger has a number of virtual methods that are instantiated
in apt-pkg/deb/dpkgpm.cc . These methods are protected in the
packagemaneger class :
// The Actual installation implementation
virtual bool Install(PkgIterator /*Pkg*/,string /*File*/) {return false;};
virtual bool Configure(PkgIterator /*Pkg*/) {return false;};
virtual bool Remove(PkgIterator /*Pkg*/,bool /*Purge*/=false) {return false;};
virtual bool Go(int statusFd=-1) {return true;};
virtual void Reset() {};
the packagemaneger object is created in cmdline/apt-get.cc by a call to
this function:
SPtr<pkgPackageManager> PM= _system->CreatePM(Cache);
In apt-get, if "Simulated" is specified, the package manager will be
instantiated as a pkgSimulate object (that is a concrete implementation of
packageManager ).
The problem is that this class does not save the installation plan anywhere it only
prints it on screen.
My proposal of integration will be very simple. We can create a new subclass of
packageManager , say pkgPlan that will implement install, configure, remove,
etc and instead of printing on stdout the actions as in pkgSimulate, will save
this information in a list that can be for our purposes. Since the dpkg
commands are built in yet another packageManager object derived from
apt-pkg/deb/dpkgpm.cc I don't think it will be easy to get a trace of what dpkg
would have done if invoked (even this would be great...). But for the moment I
think the list Inst/Conf/Rem will be enough. The real list of dpkg commands
can be obtained (not in simulation mode) with the debug variable:
Debug::pkgDPkgPM=true
In the long term maybe the correct solution to all this mambo-jumbo would be to
implement the packageManager bindings as a virtual class with a default on dpkg
as it is now, but giving the possibility to override from the python side the
Install/Configure/Remove/Go functions in order to hack into the pkgmanager
algorithm cleanly. I'm not sure this is possible given the current architecture...
As it is the OrderList class will allow to easily reimplement the backend and
in theory to completely bypass the apt-pkg/deb/dpkgpm.cc and perform the
installation via another package manager, say rpm...
thanks again for your great support.
Julian Andres Klode
> In the long term maybe the correct solution to all this mambo-jumbo would be to
> implement the packageManager bindings as a virtual class with a default on dpkg
> as it is now, but giving the possibility to override from the python side the
> Install/Configure/Remove/Go functions in order to hack into the pkgmanager
> algorithm cleanly. I'm not sure this is possible given the current architecture...
python-apt though, so we may need a slightly different solution on the C
++ side, but the Python side works. I am also attaching an example show
casing this.
I try to get this PackageManager feature implemented more cleanly
tomorrow. Until then, test the patch!