Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Re: Please test gzip -9n - related to dpkg with multiarch support
2 views
Skip to first unread message
Neil Williams
Feb 7, 2012, 5:10:03 PM2/7/12
to
On Tue, 07 Feb 2012 19:11:16 +0100
Michael Biebl <bi...@debian.org> wrote:
> On 07.02.2012 18:07, Joey Hess wrote:
> > Neil Williams wrote:
> >> I'd like to ask for some help with a bug which is tripping up my tests
> >> with the multiarch-aware dpkg from experimental - #647522 -
> >> non-deterministic behaviour of gzip -9n.
> >
> > pristine-tar hat tricks[1] aside, none of gzip, bzip2, xz are required
> > to always produce the same compressed file for a given input file, and I
> > can tell you from experience that there is a wide amount of variation. If
> > multiarch requires this, then its design is at worst broken, and at
> > best, there will be a lot of coordination pain every time there is a
> > new/different version of any of these that happens to compress slightly
> > differently.
Exactly. I'm not convinced that this is fixable at the gzip level, nor
is it likely to be fixable by the trauma of changing from gzip to
something else. That would be pointless.
What matters, to me, is that package installations do not fail
somewhere down the dependency chain in ways which are difficult to fix.
Compression is used to save space, not to provide unique identification
of file contents. As it is now clear that the compression is getting in
the way of dealing with files which are (in terms of their actual
*usable* content) identical, then the compression needs to be taken out
of the comparison operation. Where the checksum matches that's all well
and good (problems with md5sum collisions aside), where it does not
match then dpkg cannot deem that the files conflict without creating a
checksum based on the decompressed content of the two files. A checksum
failure of a compressed file is clearly unreliable and will generate
dozens of unreproducible bugs.
MultiArch has many benefits but saving space is not why MultiArch
exists and systems which will use MultiArch in anger will not be likely
to be short of either RAM or swap space. Yes, the machines which are
*targeted* by the builds which occur as a result of having MultiArch
available for Emdebian will definitely be aimed at "low resource"
devices but those devices do NOT need to actually use MultiArch
themselves. In the parlance of --build, --host and autotools, MultiArch
is a build tool, not a host mechanism. If you've got the resources to
cross-build something, you have the resources to checksum the
decompressed content of some files.
As far as having MultiArch to install non-free i386 on amd64, it is
less of a problem simply because the number of packages installed as
MultiArch packages is likely to be a lot less. Even so, although the
likelihood drops, the effect of one of these collisions getting through
is the same.
> This seems to be a rather common problem as evindenced by e.g.
>
> https://bugs.launchpad.net/ubuntu/+source/clutter-1.0/+bug/901522
> https://bugs.launchpad.net/ubuntu/+source/libtasn1-3/+bug/889303
> https://bugs.launchpad.net/ubuntu/oneiric/+source/pam/+bug/871083
See the number of .gz files in this list:
http://people.debian.org/~jwilk/multi-arch/same-md5sums.txt
> In Ubuntu they started to work-around that by excluding random files
> from being compressed. So far I refused to add those hacks to the Debian
> package as this needs to be addressed properly.
Maybe the way to solve this properly is to remove compression from the
uniqueness check - compare the contents of the file in memory after
decompression. Yes, it will take longer but it is only needed when the
md5sum (which already exists) doesn't match.
The core problem is that the times when the md5sum of the compressed
file won't match are unpredictable. No workaround is going to be
reliable because there is no apparent logic to the files which become
affected and any file which was affected at libfoo0_1.2.3 could well be
completely blameless in libfoo0_1.2.3+b1.
(binNMU's aren't the answer either because that could just as easily
transfer the bug from libfoo0 to libfoo-dev and so on.)
There appears to be plenty of evidence that checksums of compressed
files are only useful until the checksums fail to match, at which point
I think dpkg will just have to fall back to decompressing the contents
in RAM / swap and doing a fresh checksum on the contents of each
contentious compressed file. If the checksums of the contents match,
the compressed file on the filesystem wins.
Anything else and Debian loses all the reproducibility which is so
important to developers and users. When I need to make a cross-building
chroot from unstable (or write a tool for others to create such
chroots), it can't randomly fail today, work tomorrow and fail
with some other package the day after.
If others agree, I think that bug #647522, currently open against gzip,
could be reassigned to dpkg and retitled to not rely on checksums for
compressed files when determining MultiArch file collisions.
--
Neil Williams
=============
http://www.linux.codehelp.co.uk/
Michael Biebl <bi...@debian.org> wrote:
> On 07.02.2012 18:07, Joey Hess wrote:
> > Neil Williams wrote:
> >> I'd like to ask for some help with a bug which is tripping up my tests
> >> with the multiarch-aware dpkg from experimental - #647522 -
> >> non-deterministic behaviour of gzip -9n.
> >
> > pristine-tar hat tricks[1] aside, none of gzip, bzip2, xz are required
> > to always produce the same compressed file for a given input file, and I
> > can tell you from experience that there is a wide amount of variation. If
> > multiarch requires this, then its design is at worst broken, and at
> > best, there will be a lot of coordination pain every time there is a
> > new/different version of any of these that happens to compress slightly
> > differently.
Exactly. I'm not convinced that this is fixable at the gzip level, nor
is it likely to be fixable by the trauma of changing from gzip to
something else. That would be pointless.
What matters, to me, is that package installations do not fail
somewhere down the dependency chain in ways which are difficult to fix.
Compression is used to save space, not to provide unique identification
of file contents. As it is now clear that the compression is getting in
the way of dealing with files which are (in terms of their actual
*usable* content) identical, then the compression needs to be taken out
of the comparison operation. Where the checksum matches that's all well
and good (problems with md5sum collisions aside), where it does not
match then dpkg cannot deem that the files conflict without creating a
checksum based on the decompressed content of the two files. A checksum
failure of a compressed file is clearly unreliable and will generate
dozens of unreproducible bugs.
MultiArch has many benefits but saving space is not why MultiArch
exists and systems which will use MultiArch in anger will not be likely
to be short of either RAM or swap space. Yes, the machines which are
*targeted* by the builds which occur as a result of having MultiArch
available for Emdebian will definitely be aimed at "low resource"
devices but those devices do NOT need to actually use MultiArch
themselves. In the parlance of --build, --host and autotools, MultiArch
is a build tool, not a host mechanism. If you've got the resources to
cross-build something, you have the resources to checksum the
decompressed content of some files.
As far as having MultiArch to install non-free i386 on amd64, it is
less of a problem simply because the number of packages installed as
MultiArch packages is likely to be a lot less. Even so, although the
likelihood drops, the effect of one of these collisions getting through
is the same.
> This seems to be a rather common problem as evindenced by e.g.
>
> https://bugs.launchpad.net/ubuntu/+source/clutter-1.0/+bug/901522
> https://bugs.launchpad.net/ubuntu/+source/libtasn1-3/+bug/889303
> https://bugs.launchpad.net/ubuntu/oneiric/+source/pam/+bug/871083
See the number of .gz files in this list:
http://people.debian.org/~jwilk/multi-arch/same-md5sums.txt
> In Ubuntu they started to work-around that by excluding random files
> from being compressed. So far I refused to add those hacks to the Debian
> package as this needs to be addressed properly.
Maybe the way to solve this properly is to remove compression from the
uniqueness check - compare the contents of the file in memory after
decompression. Yes, it will take longer but it is only needed when the
md5sum (which already exists) doesn't match.
The core problem is that the times when the md5sum of the compressed
file won't match are unpredictable. No workaround is going to be
reliable because there is no apparent logic to the files which become
affected and any file which was affected at libfoo0_1.2.3 could well be
completely blameless in libfoo0_1.2.3+b1.
(binNMU's aren't the answer either because that could just as easily
transfer the bug from libfoo0 to libfoo-dev and so on.)
There appears to be plenty of evidence that checksums of compressed
files are only useful until the checksums fail to match, at which point
I think dpkg will just have to fall back to decompressing the contents
in RAM / swap and doing a fresh checksum on the contents of each
contentious compressed file. If the checksums of the contents match,
the compressed file on the filesystem wins.
Anything else and Debian loses all the reproducibility which is so
important to developers and users. When I need to make a cross-building
chroot from unstable (or write a tool for others to create such
chroots), it can't randomly fail today, work tomorrow and fail
with some other package the day after.
If others agree, I think that bug #647522, currently open against gzip,
could be reassigned to dpkg and retitled to not rely on checksums for
compressed files when determining MultiArch file collisions.
--
Neil Williams
=============
http://www.linux.codehelp.co.uk/
Ben Hutchings
Feb 7, 2012, 5:50:01 PM2/7/12
to
But it's worse than this: even if dpkg decompresses before comparing,
debsums won't (and mustn't, for backward compatibility). So it's
potentially necessary to fix up the md5sums file for a package
installed for multiple architectures, if it contains a file that was
compressed differently.
Ben.
--
Ben Hutchings
We get into the habit of living before acquiring the habit of thinking.
- Albert Camus
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120207224...@decadent.org.uk
Russ Allbery
Feb 7, 2012, 6:00:03 PM2/7/12
to
Neil Williams <code...@debian.org> writes:
> Maybe the way to solve this properly is to remove compression from the
> uniqueness check - compare the contents of the file in memory after
> decompression. Yes, it will take longer but it is only needed when the
> md5sum (which already exists) doesn't match.
Another possible solution is to just give any package an implicit Replaces
> Maybe the way to solve this properly is to remove compression from the
> uniqueness check - compare the contents of the file in memory after
> decompression. Yes, it will take longer but it is only needed when the
> md5sum (which already exists) doesn't match.
(possibly constrained to /usr/share/doc) on any other package with the
same name and version and a different architecture. This isn't as
defensive, in that it doesn't catch legitimate bugs where someone has made
a mistake and the packages contain different contents, but it also solves
the binNMU issue (well, "solves"; the changelog will randomly swap back
and forth between the packages, but I'm having a hard time being convinced
this is a huge problem).
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Henrique de Moraes Holschuh
Feb 7, 2012, 6:50:02 PM2/7/12
to
On Tue, 07 Feb 2012, Ben Hutchings wrote:
> But it's worse than this: even if dpkg decompresses before comparing,
> debsums won't (and mustn't, for backward compatibility). So it's
Maybe you can switch to sha256 and add the new functionality while at
> But it's worse than this: even if dpkg decompresses before comparing,
> debsums won't (and mustn't, for backward compatibility). So it's
it? Detect which mode (md5sum raw, sha256 uncompress) by the size of
the hash. Old debsums won't work with the new files, but is that really
a problem? That's what stable updates and backports are for...
--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Lars Wirzenius
Feb 8, 2012, 3:00:01 AM2/8/12
to
On Tue, Feb 07, 2012 at 10:49:23PM +0000, Ben Hutchings wrote:
> But it's worse than this: even if dpkg decompresses before comparing,
> debsums won't (and mustn't, for backward compatibility). So it's
> potentially necessary to fix up the md5sums file for a package
> installed for multiple architectures, if it contains a file that was
> compressed differently.
I'm uncomfortable with the idea of checking checksums only for
> But it's worse than this: even if dpkg decompresses before comparing,
> debsums won't (and mustn't, for backward compatibility). So it's
> potentially necessary to fix up the md5sums file for a package
> installed for multiple architectures, if it contains a file that was
> compressed differently.
uncompressed data. Compressed files have headers, and at least for
some formats, it seems those headers can contain essentially
arbitrary data. This allows compressed files to be modified in
rather significant ways, without debsums noticing, if debsums
uncompresses before comparing.
Further, uncompressors have the potential for security problems.
See https://bugzilla.redhat.com/show_bug.cgi?id=CVE-2009-2624 for example.
In other words: debsums needs to decompress to verify that no
files have been tampered with, but doing so can invoke an attack.
Such an attack may be unlikely, but it would seem to be a better design
to not open up the possibility for it.
--
http://www.kickstarter.com/projects/docstory/mix-1-2-albanian
Neil Williams
Feb 8, 2012, 5:30:02 AM2/8/12
to
On Wed, 8 Feb 2012 02:52:52 +0200
Riku Voipio <riku....@iki.fi> wrote:
> If it turns out not reasonable to expect the compression results to be
> identical, we should probably look into using dpkg --path-exclude= with
> /usr/share/{doc,man,info}/* when installing foreign-architecture packages.
That would be a suitable alternative to decompression checksums.
It sounds like an implicit Replaces: on the same package of any other
architecture for Multi-Arch: same packages limited
to /usr/share/{doc,man,info}/*.
> Very few Multi-Arch: same packages need to install identical compressed
> files outside these directories. In case it happens, the package needs to
> use multiarch paths or split files to -common package.
It's not that ugly - with a few looks at the list of problematic files.
e.g. libslang2-dev, in common with a number of other -dev packages,
includes a few example files in the -dev instead of using a -doc
package. The compression of those files causes conflicts in the -dev
package, at which point creating a -doc package doesn't seem that bad
an idea. Other options would be to not compress example files when
packaged inside -dev packages - after all, if the example files are
large enough for a lack of compression to matter, the examples should
be in a -doc package.
http://people.debian.org/~jwilk/multi-arch/same-md5sums.txt
> The ugliness of this
> solution is that the "specialness" of /usr/share/doc and others needs to
> embedded into the package system somewhere.
Packages can use multiarch paths for their own files, but there are
currently 80 occurrences of changelog.Debian.gz in the list of
problematic files. dpkg needs to handle that, packages have no option.
I'm wondering if /usr/share/{doc,man,info}/* is the right pattern.
Maybe it really is just /usr/share/*.
After all, this is how cross/foreign architecture packages have
*always* been handled in Debian via dpkg-cross. Nothing in /usr/share/
matters for a cross package created by dpkg-cross (with the possible
exception of /usr/share/pkg-config which was always anachronistic). Some
template files are added but the package name includes the
architecture, so these files are effectively in multiarch paths.
There is nothing useful in /usr/share of a Multiarch: same package
when installed as foreign architecture package. Emdebian & dpkg-cross
have proved that by having nothing else until Multi-Arch. Anything you
might need is in the native architecture package, so the best thing to
do is widen the implicit exclusion to all of /usr/share in the incoming
Multi-Arch: same package.
In the list, the only listings in the above file which are not
in /usr/share do look like bugs:
usr/bin/kvirc
usr/bin/croco-0.6-config
usr/bin/croco-0.6-config
usr/include/dspam/auto-config.h
usr/include/isl/stdint.h
usr/bin/magics-config
usr/include/OGRE/OgreBuildSettings.h
usr/include/pci/config.h
usr/lib/pkgconfig/popt.pc
usr/bin/ppl_pl
usr/bin/ppl-config
usr/include/ppl.hh
usr/include/ppl_c.h
usr/bin/ppl-config
usr/include/ppl.hh
usr/include/ppl_c.h
usr/lib/sasl2/berkeley_db.txt
usr/lib/libwrap.a
usr/include/XdmfConfig.h
usr/bin/whiptail
usr/lib/pkgconfig/popt.pc - needs to be a multiarch path
usr/bin/* is just wrong - bug reports invited.
usr/include/* means that the package concerned needs to use a multiarch
path for that include file(s).
That leaves:
usr/lib/sasl2/berkeley_db.txt
usr/lib/libwrap.a
.a files need multiarch paths, clearly.
So, apart from /usr/share which I can't see as important for
Multi-Arch: same packages, the list of remaining conflicts are bugs and
the gzip bug doesn't matter anymore.
Riku Voipio <riku....@iki.fi> wrote:
> If it turns out not reasonable to expect the compression results to be
> identical, we should probably look into using dpkg --path-exclude= with
> /usr/share/{doc,man,info}/* when installing foreign-architecture packages.
That would be a suitable alternative to decompression checksums.
It sounds like an implicit Replaces: on the same package of any other
architecture for Multi-Arch: same packages limited
to /usr/share/{doc,man,info}/*.
> Very few Multi-Arch: same packages need to install identical compressed
> files outside these directories. In case it happens, the package needs to
> use multiarch paths or split files to -common package.
It's not that ugly - with a few looks at the list of problematic files.
e.g. libslang2-dev, in common with a number of other -dev packages,
includes a few example files in the -dev instead of using a -doc
package. The compression of those files causes conflicts in the -dev
package, at which point creating a -doc package doesn't seem that bad
an idea. Other options would be to not compress example files when
packaged inside -dev packages - after all, if the example files are
large enough for a lack of compression to matter, the examples should
be in a -doc package.
http://people.debian.org/~jwilk/multi-arch/same-md5sums.txt
> The ugliness of this
> solution is that the "specialness" of /usr/share/doc and others needs to
> embedded into the package system somewhere.
Packages can use multiarch paths for their own files, but there are
currently 80 occurrences of changelog.Debian.gz in the list of
problematic files. dpkg needs to handle that, packages have no option.
I'm wondering if /usr/share/{doc,man,info}/* is the right pattern.
Maybe it really is just /usr/share/*.
After all, this is how cross/foreign architecture packages have
*always* been handled in Debian via dpkg-cross. Nothing in /usr/share/
matters for a cross package created by dpkg-cross (with the possible
exception of /usr/share/pkg-config which was always anachronistic). Some
template files are added but the package name includes the
architecture, so these files are effectively in multiarch paths.
There is nothing useful in /usr/share of a Multiarch: same package
when installed as foreign architecture package. Emdebian & dpkg-cross
have proved that by having nothing else until Multi-Arch. Anything you
might need is in the native architecture package, so the best thing to
do is widen the implicit exclusion to all of /usr/share in the incoming
Multi-Arch: same package.
In the list, the only listings in the above file which are not
in /usr/share do look like bugs:
usr/bin/kvirc
usr/bin/croco-0.6-config
usr/bin/croco-0.6-config
usr/include/dspam/auto-config.h
usr/include/isl/stdint.h
usr/bin/magics-config
usr/include/OGRE/OgreBuildSettings.h
usr/include/pci/config.h
usr/lib/pkgconfig/popt.pc
usr/bin/ppl_pl
usr/bin/ppl-config
usr/include/ppl.hh
usr/include/ppl_c.h
usr/bin/ppl-config
usr/include/ppl.hh
usr/include/ppl_c.h
usr/lib/sasl2/berkeley_db.txt
usr/lib/libwrap.a
usr/include/XdmfConfig.h
usr/bin/whiptail
usr/lib/pkgconfig/popt.pc - needs to be a multiarch path
usr/bin/* is just wrong - bug reports invited.
usr/include/* means that the package concerned needs to use a multiarch
path for that include file(s).
That leaves:
usr/lib/sasl2/berkeley_db.txt
usr/lib/libwrap.a
.a files need multiarch paths, clearly.
So, apart from /usr/share which I can't see as important for
Multi-Arch: same packages, the list of remaining conflicts are bugs and
the gzip bug doesn't matter anymore.
Daniel Baumann
Feb 8, 2012, 6:20:02 AM2/8/12
to
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On 02/07/2012 11:04 PM, Neil Williams wrote:
> I'm not convinced that this is fixable at the gzip level, nor is it
> likely to be fixable by the trauma of changing from gzip to
> something else.
while the original point of not considering compressors that are not
producing identical output across all archs in dpkgs multi-arch
implementation still stands, it might be worth noting (and at least
jftr) that lzip does not suffer from that problem in the first place.
- --
Address: Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email: daniel....@progress-technologies.net
Internet: http://people.progress-technologies.net/~daniel.baumann/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8yWOYACgkQ+C5cwEsrK57kRACg4LIBEB+Yn6bMC9E+xybh/doX
FJAAn2Ufes5sZTMn4XHYQ2fr5ja/w6W6
=qbRU
-----END PGP SIGNATURE-----
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4F3258E6...@progress-technologies.net
Hash: SHA1
On 02/07/2012 11:04 PM, Neil Williams wrote:
> I'm not convinced that this is fixable at the gzip level, nor is it
> likely to be fixable by the trauma of changing from gzip to
> something else.
producing identical output across all archs in dpkgs multi-arch
implementation still stands, it might be worth noting (and at least
jftr) that lzip does not suffer from that problem in the first place.
- --
Address: Daniel Baumann, Donnerbuehlweg 3, CH-3012 Bern
Email: daniel....@progress-technologies.net
Internet: http://people.progress-technologies.net/~daniel.baumann/
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk8yWOYACgkQ+C5cwEsrK57kRACg4LIBEB+Yn6bMC9E+xybh/doX
FJAAn2Ufes5sZTMn4XHYQ2fr5ja/w6W6
=qbRU
-----END PGP SIGNATURE-----
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Wouter Verhelst
Feb 8, 2012, 8:20:03 AM2/8/12
to
On Tue, Feb 07, 2012 at 10:04:04PM +0000, Neil Williams wrote:
> Maybe the way to solve this properly is to remove compression from the
> uniqueness check - compare the contents of the file in memory after
> decompression. Yes, it will take longer but it is only needed when the
> md5sum (which already exists) doesn't match.
Actually, I think the real way to fix this properly is to not compress
> uniqueness check - compare the contents of the file in memory after
> decompression. Yes, it will take longer but it is only needed when the
> md5sum (which already exists) doesn't match.
files in the package at all.
The contents.tar.gz is already a .tar.gz, which means it's compressed.
Doubly-compressing files hardly ever nets a benefit, so we're not
compressing files for the benefit of our mirrors.
The only reason why we compress files in /usr/share/doc is so that that
directory doesn't waste too much space. If that is the case, I think it
makes much more sense for files to be packaged inside .debs
uncompressed, and (optionally) for dpkg to compress them on the fly
should the system administrator request it. It would then make much more
sense for dpkg to consider the contents of the file, rather than the
on-disk representation, and not cause this kind of issues.
As an additional benefit, this will also allow those among us (like me)
who hate having to use 'gunzip -c /usr/share/doc/foo/bar.pdf.gz >
/tmp/bar.pdf; xpdf /tmp/bar.pdf' in order to be able to read some
documentation, to just request that files are not compressed.
--
The volume of a pizza of thickness a and radius z can be described by
the following formula:
pi zz a
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 8, 2012, 8:40:03 AM2/8/12
to
Russ Allbery writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
because the other changelog wouldn't be present on the system.
One thing which no-one yet seems to have suggested is to have
multiarch:same packages put the changelog in a filename which is
distinct for each architecture. (It wouldn't have to be the triplet;
the shorter Debian arch would do.) Perhaps there are obvious reasons
(which I have missed) why this is a terrible idea, but it seems to me
that it's something we should consider.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20274.30293....@chiark.greenend.org.uk
> Another possible solution is to just give any package an implicit Replaces
> (possibly constrained to /usr/share/doc) on any other package with the
> same name and version and a different architecture. This isn't as
> defensive, in that it doesn't catch legitimate bugs where someone has made
> a mistake and the packages contain different contents, but it also solves
> the binNMU issue (well, "solves"; the changelog will randomly swap back
> and forth between the packages, but I'm having a hard time being convinced
> this is a huge problem).
Well, it does mean that you might be lacking important information
> (possibly constrained to /usr/share/doc) on any other package with the
> same name and version and a different architecture. This isn't as
> defensive, in that it doesn't catch legitimate bugs where someone has made
> a mistake and the packages contain different contents, but it also solves
> the binNMU issue (well, "solves"; the changelog will randomly swap back
> and forth between the packages, but I'm having a hard time being convinced
> this is a huge problem).
because the other changelog wouldn't be present on the system.
One thing which no-one yet seems to have suggested is to have
multiarch:same packages put the changelog in a filename which is
distinct for each architecture. (It wouldn't have to be the triplet;
the shorter Debian arch would do.) Perhaps there are obvious reasons
(which I have missed) why this is a terrible idea, but it seems to me
that it's something we should consider.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Guillem Jover
Feb 8, 2012, 11:20:01 AM2/8/12
to
On Wed, 2012-02-08 at 13:19:17 +0000, Ian Jackson wrote:
> Well, it does mean that you might be lacking important information
> because the other changelog wouldn't be present on the system.
While the implicit Replaces seems the easy way out, it just seems even
> Well, it does mean that you might be lacking important information
> because the other changelog wouldn't be present on the system.
more fragile than the shared files approach.
And while the binNMU changelog issues might seem like a corner case,
it's just a symptom of something that's not quite right. And after
this was brought up again I started considering that the shared file
approach might have been flawed afterall, even if it might have seemed
neat at the time (it's one of the reasons that part of the code has
not been merged yet). The main reason it was enviaged was to handle the
changelog and copyright files and to avoid needing to introduce an
additional common package per source, for just those two/three files.
As a side remark, I think at least those two are actual package
metadata and do belong in the .deb control member [0], and as such in
the dpkg database. But that's for discussion on another time, because
that would not fix the issue as upstream changelogs do conflict too,
for example.
<http://lists.debian.org/debian-dpkg/2011/09/msg00029.html>
> One thing which no-one yet seems to have suggested is to have
> multiarch:same packages put the changelog in a filename which is
> distinct for each architecture. (It wouldn't have to be the triplet;
> the shorter Debian arch would do.) Perhaps there are obvious reasons
> (which I have missed) why this is a terrible idea, but it seems to me
> that it's something we should consider.
completely, and /usr/share/doc/ files for “Multi-Arch: same” packages
be installed under /usr/share/doc/pkgname:arch/. This would solve all
these problems in a clean way for the common case with just the two or
three mandated files (changelog, changelog.Debian and copyright), if
a package provides lots more files then they should be split anyway
into either a libfooN-common libfoo-doc, or similar. And finally this
would not be really confusing, given that one of the last interface
changes was to make all dpkg output for all “Multi-Arch: same”
packages be always arch-qualified.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Mike Hommey
Feb 8, 2012, 11:30:02 AM2/8/12
to
lintian overrides, reportbug presubj and scripts, etc. ?
Mike
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Guillem Jover
Feb 8, 2012, 12:00:01 PM2/8/12
to
On Wed, 2012-02-08 at 17:29:22 +0100, Mike Hommey wrote:
> If you remove the shared files approach, how do you handle files like
> lintian overrides, reportbug presubj and scripts, etc. ?
The same principle that applies to all dpkg output to avoid ambiguity
> If you remove the shared files approach, how do you handle files like
> lintian overrides, reportbug presubj and scripts, etc. ?
would apply everywhere, whenever there's a “Multi-Arch: same” package
name that needs to be unambiguous, it just always gets arch-qualified.
The rest would stay as they are.
So, at least for lintian and reportbug, given that these file/dir names
are package name based they would just get arch-qualified when needed.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 8, 2012, 12:20:02 PM2/8/12
to
Guillem Jover writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
generalised it. It doesn't seem like an unreasonable idea to me.
Obviously it would mean that some (Debian-specific) software which
currently doesn't need to be multiarch-aware would need to be taught
about these new directory names. But that seems like a reasonable
price to pay for solving the varying compressed shared files problem.
Another relevant question is whether there are other files that are
shared, and which don't want to move, besides ones in /usr/share/doc.
I haven't been following this in detail but if there are then we may
need to retain the possibility to have actually-identical shared
files.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20274.43538....@chiark.greenend.org.uk
> On Wed, 2012-02-08 at 13:19:17 +0000, Ian Jackson wrote:
> > One thing which no-one yet seems to have suggested is to have
> > multiarch:same packages put the changelog in a filename which is
> > distinct for each architecture. (It wouldn't have to be the triplet;
> > the shorter Debian arch would do.) Perhaps there are obvious reasons
> > (which I have missed) why this is a terrible idea, but it seems to me
> > that it's something we should consider.
>
> > multiarch:same packages put the changelog in a filename which is
> > distinct for each architecture. (It wouldn't have to be the triplet;
> > the shorter Debian arch would do.) Perhaps there are obvious reasons
> > (which I have missed) why this is a terrible idea, but it seems to me
> > that it's something we should consider.
>
> Instead of this, I'd rather see the shared files approach just dropped
> completely, and /usr/share/doc/ files for “Multi-Arch: same” packages
> be installed under /usr/share/doc/pkgname:arch/.
Right, that's kind of what I was suggesting although you've
> completely, and /usr/share/doc/ files for “Multi-Arch: same” packages
> be installed under /usr/share/doc/pkgname:arch/.
generalised it. It doesn't seem like an unreasonable idea to me.
Obviously it would mean that some (Debian-specific) software which
currently doesn't need to be multiarch-aware would need to be taught
about these new directory names. But that seems like a reasonable
price to pay for solving the varying compressed shared files problem.
Another relevant question is whether there are other files that are
shared, and which don't want to move, besides ones in /usr/share/doc.
I haven't been following this in detail but if there are then we may
need to retain the possibility to have actually-identical shared
files.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Jonathan Nieder
Feb 8, 2012, 12:30:02 PM2/8/12
to
Ian Jackson wrote:
> Another relevant question is whether there are other files that are
> shared, and which don't want to move, besides ones in /usr/share/doc.
One example is header files in /usr/include, from -dev packages. In
> Another relevant question is whether there are other files that are
> shared, and which don't want to move, besides ones in /usr/share/doc.
the simple examples I've seen, putting them in /usr/include/<triplet>,
works fine. It is always possible to split off a separate "Multiarch:
foreign" -dev-common package if needed in order to save space.
Another example is manpages, also in -dev packages. That's more
fussy.
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120208172627.GA6712@burratino
Riku Voipio
Feb 8, 2012, 1:00:02 PM2/8/12
to
On Wed, Feb 08, 2012 at 05:56:11PM +0100, Guillem Jover wrote:
> On Wed, 2012-02-08 at 17:29:22 +0100, Mike Hommey wrote:
> > If you remove the shared files approach, how do you handle files like
> > lintian overrides, reportbug presubj and scripts, etc. ?
>
> The same principle that applies to all dpkg output to avoid ambiguity
> would apply everywhere, whenever there's a “Multi-Arch: same” package
> name that needs to be unambiguous, it just always gets arch-qualified.
> The rest would stay as they are.
That is a major waste of space of having multiple copies of identical files
> On Wed, 2012-02-08 at 17:29:22 +0100, Mike Hommey wrote:
> > If you remove the shared files approach, how do you handle files like
> > lintian overrides, reportbug presubj and scripts, etc. ?
>
> The same principle that applies to all dpkg output to avoid ambiguity
> would apply everywhere, whenever there's a “Multi-Arch: same” package
> name that needs to be unambiguous, it just always gets arch-qualified.
> The rest would stay as they are.
with different arch-qualified names. Is that really better architecture to
have multiple copies of identical files on user systems?
> So, at least for lintian and reportbug, given that these file/dir names
> are package name based they would just get arch-qualified when needed.
to split the M-A: same libfoo-dev packages to libfoo-dev-common in order to
avoid overwriting /usr/include contents and /usr/bin/foo-config binaries. Our
packages are already heavily split slowing down Packages.gz downloads and
all other apt operations.
Riku
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 8, 2012, 2:50:01 PM2/8/12
to
Guillem Jover <gui...@debian.org> writes:
> On Wed, 2012-02-08 at 13:19:17 +0000, Ian Jackson wrote:
>> Well, it does mean that you might be lacking important information
>> because the other changelog wouldn't be present on the system.
> While the implicit Replaces seems the easy way out, it just seems even
> more fragile than the shared files approach.
Yes, this is definitely true. I was mentioning it as an easy way out, but
> On Wed, 2012-02-08 at 13:19:17 +0000, Ian Jackson wrote:
>> Well, it does mean that you might be lacking important information
>> because the other changelog wouldn't be present on the system.
> While the implicit Replaces seems the easy way out, it just seems even
> more fragile than the shared files approach.
it's aesthetically unappealing.
> And while the binNMU changelog issues might seem like a corner case,
> it's just a symptom of something that's not quite right.
time with linked doc directories. I'd like to prohibit them in more cases
so that we get the binNMU changelogs on disk.
> Instead of this, I'd rather see the shared files approach just dropped
> completely, and /usr/share/doc/ files for “Multi-Arch: same” packages be
> installed under /usr/share/doc/pkgname:arch/. This would solve all these
> problems in a clean way for the common case with just the two or three
> mandated files (changelog, changelog.Debian and copyright), if a package
> provides lots more files then they should be split anyway into either a
> libfooN-common libfoo-doc, or similar. And finally this would not be
> really confusing, given that one of the last interface changes was to
> make all dpkg output for all “Multi-Arch: same” packages be always
> arch-qualified.
UI perspective. That's going to be a change historically from where we've
told users to look, and it's a little awkward. But, thinking it over, the
set of packages that we're talking about is fairly limited.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87liodr...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Steve Langasek
Feb 8, 2012, 7:00:02 PM2/8/12
to
On Wed, Feb 08, 2012 at 10:22:17AM +0000, Neil Williams wrote:
> After all, this is how cross/foreign architecture packages have
> *always* been handled in Debian via dpkg-cross. Nothing in /usr/share/
> matters for a cross package created by dpkg-cross (with the possible
> exception of /usr/share/pkg-config which was always anachronistic). Some
> template files are added but the package name includes the
> architecture, so these files are effectively in multiarch paths.
> There is nothing useful in /usr/share of a Multiarch: same package
> when installed as foreign architecture package. Emdebian & dpkg-cross
> have proved that by having nothing else until Multi-Arch. Anything you
> might need is in the native architecture package, so the best thing to
> do is widen the implicit exclusion to all of /usr/share in the incoming
> Multi-Arch: same package.
The unfounded assumption here is that you will always install a foreign-arch
> After all, this is how cross/foreign architecture packages have
> *always* been handled in Debian via dpkg-cross. Nothing in /usr/share/
> matters for a cross package created by dpkg-cross (with the possible
> exception of /usr/share/pkg-config which was always anachronistic). Some
> template files are added but the package name includes the
> architecture, so these files are effectively in multiarch paths.
> There is nothing useful in /usr/share of a Multiarch: same package
> when installed as foreign architecture package. Emdebian & dpkg-cross
> have proved that by having nothing else until Multi-Arch. Anything you
> might need is in the native architecture package, so the best thing to
> do is widen the implicit exclusion to all of /usr/share in the incoming
> Multi-Arch: same package.
M-A: same package together with the native-arch version. If I install
libaudio2:i386 because I want to play a game that's only available as a
32-bit binary and has this lib as a dependency, and nothing else on my
system uses libaudio2, I still expect to get /usr/share/libaudio2/AuErrorDB
installed.
In general, anything that introduces assymetric handling between native and
foreign arch packages at the dpkg level is probably going to be a bad idea.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slan...@ubuntu.com vor...@debian.org
Russ Allbery
Feb 8, 2012, 8:00:02 PM2/8/12
to
Steve Langasek <vor...@debian.org> writes:
> The unfounded assumption here is that you will always install a
> foreign-arch M-A: same package together with the native-arch version.
> If I install libaudio2:i386 because I want to play a game that's only
> available as a 32-bit binary and has this lib as a dependency, and
> nothing else on my system uses libaudio2, I still expect to get
> /usr/share/libaudio2/AuErrorDB installed.
How is that not a serious policy violation already? AuErrorDB isn't
> The unfounded assumption here is that you will always install a
> foreign-arch M-A: same package together with the native-arch version.
> If I install libaudio2:i386 because I want to play a game that's only
> available as a 32-bit binary and has this lib as a dependency, and
> nothing else on my system uses libaudio2, I still expect to get
> /usr/share/libaudio2/AuErrorDB installed.
versioned with the SONAME, so libaudio2 and libaudio3 would not be
coinstallable.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Steve Langasek
Feb 8, 2012, 8:10:02 PM2/8/12
to
On Wed, Feb 08, 2012 at 04:55:02PM -0800, Russ Allbery wrote:
> Steve Langasek <vor...@debian.org> writes:
>
> > The unfounded assumption here is that you will always install a
> > foreign-arch M-A: same package together with the native-arch version.
> > If I install libaudio2:i386 because I want to play a game that's only
> > available as a 32-bit binary and has this lib as a dependency, and
> > nothing else on my system uses libaudio2, I still expect to get
> > /usr/share/libaudio2/AuErrorDB installed.
> How is that not a serious policy violation already? AuErrorDB isn't
> versioned with the SONAME, so libaudio2 and libaudio3 would not be
> coinstallable.
Because libaudio2 is in the directory name.
> Steve Langasek <vor...@debian.org> writes:
>
> > The unfounded assumption here is that you will always install a
> > foreign-arch M-A: same package together with the native-arch version.
> > If I install libaudio2:i386 because I want to play a game that's only
> > available as a 32-bit binary and has this lib as a dependency, and
> > nothing else on my system uses libaudio2, I still expect to get
> > /usr/share/libaudio2/AuErrorDB installed.
> How is that not a serious policy violation already? AuErrorDB isn't
> versioned with the SONAME, so libaudio2 and libaudio3 would not be
> coinstallable.
Also, it's not a policy violation for a library package to contain files
that don't have sensibly versioned names; it's only a policy violation for
the name to not change on soname bump. So even if this were called
/usr/share/AuErrorDB, it could be changed to /usr/share/libaudio3/AuErrorDB
on soname change and still be compliant.
Russ Allbery
Feb 8, 2012, 8:20:01 PM2/8/12
to
Steve Langasek <vor...@debian.org> writes:
> On Wed, Feb 08, 2012 at 04:55:02PM -0800, Russ Allbery wrote:
>> Steve Langasek <vor...@debian.org> writes:
>>> The unfounded assumption here is that you will always install a
>>> foreign-arch M-A: same package together with the native-arch version.
>>> If I install libaudio2:i386 because I want to play a game that's only
>>> available as a 32-bit binary and has this lib as a dependency, and
>>> nothing else on my system uses libaudio2, I still expect to get
>>> /usr/share/libaudio2/AuErrorDB installed.
>> How is that not a serious policy violation already? AuErrorDB isn't
>> versioned with the SONAME, so libaudio2 and libaudio3 would not be
>> coinstallable.
> Because libaudio2 is in the directory name.
Oh, duh. Sorry, I'm just blind.
> On Wed, Feb 08, 2012 at 04:55:02PM -0800, Russ Allbery wrote:
>> Steve Langasek <vor...@debian.org> writes:
>>> The unfounded assumption here is that you will always install a
>>> foreign-arch M-A: same package together with the native-arch version.
>>> If I install libaudio2:i386 because I want to play a game that's only
>>> available as a 32-bit binary and has this lib as a dependency, and
>>> nothing else on my system uses libaudio2, I still expect to get
>>> /usr/share/libaudio2/AuErrorDB installed.
>> How is that not a serious policy violation already? AuErrorDB isn't
>> versioned with the SONAME, so libaudio2 and libaudio3 would not be
>> coinstallable.
> Because libaudio2 is in the directory name.
> Also, it's not a policy violation for a library package to contain files
> that don't have sensibly versioned names; it's only a policy violation
> for the name to not change on soname bump. So even if this were called
> /usr/share/AuErrorDB, it could be changed to
> /usr/share/libaudio3/AuErrorDB on soname change and still be compliant.
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87ty30i...@windlord.stanford.edu
Goswin von Brederlow
Feb 9, 2012, 7:40:02 AM2/9/12
to
Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> Russ Allbery writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
>> Another possible solution is to just give any package an implicit Replaces
>> (possibly constrained to /usr/share/doc) on any other package with the
>> same name and version and a different architecture. This isn't as
>> defensive, in that it doesn't catch legitimate bugs where someone has made
>> a mistake and the packages contain different contents, but it also solves
>> the binNMU issue (well, "solves"; the changelog will randomly swap back
>> and forth between the packages, but I'm having a hard time being convinced
>> this is a huge problem).
>
> Well, it does mean that you might be lacking important information
> because the other changelog wouldn't be present on the system.
>
> One thing which no-one yet seems to have suggested is to have
> multiarch:same packages put the changelog in a filename which is
> distinct for each architecture. (It wouldn't have to be the triplet;
> the shorter Debian arch would do.) Perhaps there are obvious reasons
> (which I have missed) why this is a terrible idea, but it seems to me
> that it's something we should consider.
>
> Ian.
Or dpkg could do that for you. At least for files in /usr/share/doc when
> Russ Allbery writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
>> Another possible solution is to just give any package an implicit Replaces
>> (possibly constrained to /usr/share/doc) on any other package with the
>> same name and version and a different architecture. This isn't as
>> defensive, in that it doesn't catch legitimate bugs where someone has made
>> a mistake and the packages contain different contents, but it also solves
>> the binNMU issue (well, "solves"; the changelog will randomly swap back
>> and forth between the packages, but I'm having a hard time being convinced
>> this is a huge problem).
>
> Well, it does mean that you might be lacking important information
> because the other changelog wouldn't be present on the system.
>
> One thing which no-one yet seems to have suggested is to have
> multiarch:same packages put the changelog in a filename which is
> distinct for each architecture. (It wouldn't have to be the triplet;
> the shorter Debian arch would do.) Perhaps there are obvious reasons
> (which I have missed) why this is a terrible idea, but it seems to me
> that it's something we should consider.
>
> Ian.
there is a collision.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 9, 2012, 8:00:02 AM2/9/12
to
Goswin von Brederlow writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
packages contain the arch-specific filenames. After all a
multiarch:same package knows that it is, and can DTRT.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20275.48825....@chiark.greenend.org.uk
> Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> > One thing which no-one yet seems to have suggested is to have
> > multiarch:same packages put the changelog in a filename which is
> > distinct for each architecture. (It wouldn't have to be the triplet;
> > the shorter Debian arch would do.) Perhaps there are obvious reasons
> > (which I have missed) why this is a terrible idea, but it seems to me
> > that it's something we should consider.
>
> > One thing which no-one yet seems to have suggested is to have
> > multiarch:same packages put the changelog in a filename which is
> > distinct for each architecture. (It wouldn't have to be the triplet;
> > the shorter Debian arch would do.) Perhaps there are obvious reasons
> > (which I have missed) why this is a terrible idea, but it seems to me
> > that it's something we should consider.
>
> Or dpkg could do that for you. At least for files in /usr/share/doc when
> there is a collision.
Urgh, I think this is a really ugly idea, compared to just having the
> there is a collision.
packages contain the arch-specific filenames. After all a
multiarch:same package knows that it is, and can DTRT.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 9, 2012, 9:00:03 AM2/9/12
to
Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> Goswin von Brederlow writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
>> Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
>> > One thing which no-one yet seems to have suggested is to have
>> > multiarch:same packages put the changelog in a filename which is
>> > distinct for each architecture. (It wouldn't have to be the triplet;
>> > the shorter Debian arch would do.) Perhaps there are obvious reasons
>> > (which I have missed) why this is a terrible idea, but it seems to me
>> > that it's something we should consider.
>>
>> Or dpkg could do that for you. At least for files in /usr/share/doc when
>> there is a collision.
>
> Urgh, I think this is a really ugly idea, compared to just having the
> packages contain the arch-specific filenames. After all a
> multiarch:same package knows that it is, and can DTRT.
>
> Ian.
Changing the name in the package would break tools that rely on the
> Goswin von Brederlow writes ("Re: Please test gzip -9n - related to dpkg with multiarch support"):
>> Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
>> > One thing which no-one yet seems to have suggested is to have
>> > multiarch:same packages put the changelog in a filename which is
>> > distinct for each architecture. (It wouldn't have to be the triplet;
>> > the shorter Debian arch would do.) Perhaps there are obvious reasons
>> > (which I have missed) why this is a terrible idea, but it seems to me
>> > that it's something we should consider.
>>
>> Or dpkg could do that for you. At least for files in /usr/share/doc when
>> there is a collision.
>
> Urgh, I think this is a really ugly idea, compared to just having the
> packages contain the arch-specific filenames. After all a
> multiarch:same package knows that it is, and can DTRT.
>
> Ian.
name (like packages.debian.org extracting the Changelog). Also ugly.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 9, 2012, 4:10:01 PM2/9/12
to
Goswin von Brederlow <goswi...@web.de> writes:
> Changing the name in the package would break tools that rely on the name
> (like packages.debian.org extracting the Changelog). Also ugly.
We control the tools; we can change the tools. Multiarch is a big deal.
> Changing the name in the package would break tools that rely on the name
> (like packages.debian.org extracting the Changelog). Also ugly.
We weren't going to get through this without changing some tools. (One
should not read that as my support of this specific alternative, as I've
not decided there yet, but in general I think it's fair game to change our
tools to support multiarch.)
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87k43vu...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 14, 2012, 1:50:01 AM2/14/12
to
There's been a lot of discussion of this, but it seems to have been fairly
inconclusive. We need to decide what we're doing, if anything, for wheezy
fairly soon, so I think we need to try to drive this discussion to some
concrete conclusions.
First, Steve's point here is very good:
Steve Langasek <vor...@debian.org> writes:
> I guess we're looking at the same data, yet we seem to have reached
> opposite conclusions.
> - Riku reports that 33 out of 82k files have different compression when
> using current gzip vs. 10-year-old gzip. I'd be surprised if any of
> those binary packages hadn't been superseded long ago. It's not a
> guarantee, but I think the risks, and ultimate cost, of relying on gzip
> output to not change often and to just do sourceful rebuilds when it
> isn't are a lot smaller than if we go about manually splitting our
> packages further.
> - The cases where gzip output has been reported to not be reproducible
> seem to all boil down to a single issue with gzip being passed
> different arguments due to the unreproducible nature of *find*'s
> output. A patch has been made available already on the bug, and this
> patch seems to address the instances of the problem that we've hit so
> far in the Ubuntu archive.
> Now, it's worth following up with gzip upstream about our concerns, but
> even without that, I just don't see this being problematic.
It isn't the end of the world if we have some conflicts provided that we
can detect them and can do something consistent to fix them. I'm rather
nervous about relying on reproducibility of gzip because of Joey's
experience with pristine-tar, where he does find a lot of variation in
practice, but it is true that, for the purposes of multiarch, Debian *can*
possibly construct things such that we only need to worry about our own
gzip, which does simplify the situation.
However, as we've subsequently discussed, those are not the only issues
with file overlaps between packages. So I'm going to try to summarize and
propose some possible solutions for the different issues. I'm going to
discuss these issues in order from the most consistent with a refcounting
solution to the least consistent.
1. Uncompressed files that we know are absolutely identical between
different architectures. These include arch-independent header files
that are just copied verbatim from the upstream source and data files
in textual formats or arch-independent binary formats that aren't
compressed and whose generation doesn't vary. (Symlinks are a special
case of this.) Reference counting works great for these. These also
resolve most of the file overlaps between -dev packages, and many of
the harder cases for interpackage dependencies if we split everything
out. I think it makes a lot of sense to use refcounting for these
files.
2. Files like the above but that are compressed. This is most common in
the doc directory for things like README or the upstream changelog.
Upstream man pages written directly in *roff fall into this category as
well, for -dev packages. With Steve's point above about gzip, I think
we're probably okay using refcounting for this as well.
3. Generated documentation. Here's where I think refcounting starts
failing. Man pages generated from POD may change if the version of
Perl used to generate them changes, if Pod::Simple or Pod::Man have had
a new release. Doxygen-generated HTML documentation is even more
likely to change. Many documentation generation systems will include
timestamps or other information that changes, or (even more likely)
will have minor changes in their output and formatting even if there is
nothing as obvious as a version number or timestamp.
I don't think we can use refcounting for generated documentation
produced as part of the package build process. If there is
Doxygen-generated documentation, generated man pages, or the like, I
think those have to be split into a separate arch: all package. Even
if it's just a couple of man pages. This is rather annoying, but I
think trying to use refcounting here is just too fragile.
4. Lintian overrides. I believe these should be qualified with the
architecture on any multiarch: same package so that the overrides can
vary by architecture, since this is a semi-frequent use case for
Lintian.
5. Data files that vary by architecture. This includes big-endian
vs. little-endian issues. These are simply incompatible with multiarch
as currently designed, and incompatible with the obvious variations
that I can think of, and will have to either be moved into
arch-qualified directories (with corresponding patches to the paths
from which the libraries load the data) or these packages can't be made
multiarch.
6. Debian changelogs. The actual content of these files change with
binNMUs, so these obviously can't be refcounted at all right now. We
have to do something else here, probably by generating new
binary-specific changelog files for binNMUs.
Does this seem comprehensive to everyone? Am I missing any cases?
If this is comprehensive, then I propose the following path forward, which
is a mix of the various solutions that have been discussed:
* dpkg re-adds the refcounting implementation for multiarch, but along
with a Policy requirement that packages that are multiarch must only
contain files in classes 1 and 2 above.
* All packages that want to be multiarch: same have to move all generated
documentation into a separate package unless the maintainer has very
carefully checked that the generated documentation will be byte-for-byte
identical even across minor updates of the documentation generation
tools and when run at different times.
* Lintian should recognize arch-qualified override files, and multiarch:
same packages must arch-qualify their override files. debhelper
assistance is desired for this.
* Policy prohibits arch-varying data files in multiarch: same packages
except in arch-qualified paths.
* The binNMU process is changed to add the binNMU changelog entry to an
arch-qualified file (changelog.Debian.arch, probably). We need to
figure out what this means if the package being binNMU'd has a
/usr/share/doc/<package> symlink to another package, though; it's not
obvious what to do here.
Please note that this is a bunch of work. I think the Lintian work is a
good idea regardless, and it can start independently. I think the same is
true of the binNMU changelog work, since this will address some
long-standing issues with changelog handling in some situations, including
resolving just how we're supposed to handle /usr/share/doc symlinks. But
even with those aside, this is a lot of stuff that we need to agree on,
and in some cases implement, in a fairly short timeline if this is going
to make wheezy.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/874nutnc...@windlord.stanford.edu
inconclusive. We need to decide what we're doing, if anything, for wheezy
fairly soon, so I think we need to try to drive this discussion to some
concrete conclusions.
First, Steve's point here is very good:
Steve Langasek <vor...@debian.org> writes:
> I guess we're looking at the same data, yet we seem to have reached
> opposite conclusions.
> - Riku reports that 33 out of 82k files have different compression when
> using current gzip vs. 10-year-old gzip. I'd be surprised if any of
> those binary packages hadn't been superseded long ago. It's not a
> guarantee, but I think the risks, and ultimate cost, of relying on gzip
> output to not change often and to just do sourceful rebuilds when it
> isn't are a lot smaller than if we go about manually splitting our
> packages further.
> - The cases where gzip output has been reported to not be reproducible
> seem to all boil down to a single issue with gzip being passed
> different arguments due to the unreproducible nature of *find*'s
> output. A patch has been made available already on the bug, and this
> patch seems to address the instances of the problem that we've hit so
> far in the Ubuntu archive.
> Now, it's worth following up with gzip upstream about our concerns, but
> even without that, I just don't see this being problematic.
It isn't the end of the world if we have some conflicts provided that we
can detect them and can do something consistent to fix them. I'm rather
nervous about relying on reproducibility of gzip because of Joey's
experience with pristine-tar, where he does find a lot of variation in
practice, but it is true that, for the purposes of multiarch, Debian *can*
possibly construct things such that we only need to worry about our own
gzip, which does simplify the situation.
However, as we've subsequently discussed, those are not the only issues
with file overlaps between packages. So I'm going to try to summarize and
propose some possible solutions for the different issues. I'm going to
discuss these issues in order from the most consistent with a refcounting
solution to the least consistent.
1. Uncompressed files that we know are absolutely identical between
different architectures. These include arch-independent header files
that are just copied verbatim from the upstream source and data files
in textual formats or arch-independent binary formats that aren't
compressed and whose generation doesn't vary. (Symlinks are a special
case of this.) Reference counting works great for these. These also
resolve most of the file overlaps between -dev packages, and many of
the harder cases for interpackage dependencies if we split everything
out. I think it makes a lot of sense to use refcounting for these
files.
2. Files like the above but that are compressed. This is most common in
the doc directory for things like README or the upstream changelog.
Upstream man pages written directly in *roff fall into this category as
well, for -dev packages. With Steve's point above about gzip, I think
we're probably okay using refcounting for this as well.
3. Generated documentation. Here's where I think refcounting starts
failing. Man pages generated from POD may change if the version of
Perl used to generate them changes, if Pod::Simple or Pod::Man have had
a new release. Doxygen-generated HTML documentation is even more
likely to change. Many documentation generation systems will include
timestamps or other information that changes, or (even more likely)
will have minor changes in their output and formatting even if there is
nothing as obvious as a version number or timestamp.
I don't think we can use refcounting for generated documentation
produced as part of the package build process. If there is
Doxygen-generated documentation, generated man pages, or the like, I
think those have to be split into a separate arch: all package. Even
if it's just a couple of man pages. This is rather annoying, but I
think trying to use refcounting here is just too fragile.
4. Lintian overrides. I believe these should be qualified with the
architecture on any multiarch: same package so that the overrides can
vary by architecture, since this is a semi-frequent use case for
Lintian.
5. Data files that vary by architecture. This includes big-endian
vs. little-endian issues. These are simply incompatible with multiarch
as currently designed, and incompatible with the obvious variations
that I can think of, and will have to either be moved into
arch-qualified directories (with corresponding patches to the paths
from which the libraries load the data) or these packages can't be made
multiarch.
6. Debian changelogs. The actual content of these files change with
binNMUs, so these obviously can't be refcounted at all right now. We
have to do something else here, probably by generating new
binary-specific changelog files for binNMUs.
Does this seem comprehensive to everyone? Am I missing any cases?
If this is comprehensive, then I propose the following path forward, which
is a mix of the various solutions that have been discussed:
* dpkg re-adds the refcounting implementation for multiarch, but along
with a Policy requirement that packages that are multiarch must only
contain files in classes 1 and 2 above.
* All packages that want to be multiarch: same have to move all generated
documentation into a separate package unless the maintainer has very
carefully checked that the generated documentation will be byte-for-byte
identical even across minor updates of the documentation generation
tools and when run at different times.
* Lintian should recognize arch-qualified override files, and multiarch:
same packages must arch-qualify their override files. debhelper
assistance is desired for this.
* Policy prohibits arch-varying data files in multiarch: same packages
except in arch-qualified paths.
* The binNMU process is changed to add the binNMU changelog entry to an
arch-qualified file (changelog.Debian.arch, probably). We need to
figure out what this means if the package being binNMU'd has a
/usr/share/doc/<package> symlink to another package, though; it's not
obvious what to do here.
Please note that this is a bunch of work. I think the Lintian work is a
good idea regardless, and it can start independently. I think the same is
true of the binNMU changelog work, since this will address some
long-standing issues with changelog handling in some situations, including
resolving just how we're supposed to handle /usr/share/doc symlinks. But
even with those aside, this is a lot of stuff that we need to agree on,
and in some cases implement, in a fairly short timeline if this is going
to make wheezy.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 14, 2012, 2:50:01 AM2/14/12
to
On Mon, 13 Feb 2012, Russ Allbery wrote:
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
Thanks for this.
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
> 2. Files like the above but that are compressed. This is most common in
> the doc directory for things like README or the upstream changelog.
> Upstream man pages written directly in *roff fall into this category as
> well, for -dev packages. With Steve's point above about gzip, I think
> we're probably okay using refcounting for this as well.
without downsides, it's best to move compressed files in a shared package.
Also it might be wise to relax the policy rules on compression for
multi-arch: same and to let dh_compress not compress (some) files in such
packages.
> Does this seem comprehensive to everyone? Am I missing any cases?
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
to deal with that at the dpkg-dev level but dpkg-dev is not at all
involved in installing the changelog. And I believe that the bin-nmu
process just adds a top-level entry to debian/changelog.
So the code should go to dh_installchangelogs... but it doesn't seem to be
a good idea to put the bin-nmu logic there in particular since we might
extend it (see #440094).
Somehow my suggestion is then to extend dpkg-parsechangelog to provide
the required logic to split the changelog in its bin-nmu part and its
usual content.
dpkg-parsechangelog --split-binnmu <binnmu-part-file> <remaining-part-file>
Then dh_installchangelogs could try to use this (and if it fails, fallback
to the standard changelog installation).
Does that sound sane? If yes, I can have a look at implementing this.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Philipp Kern
Feb 14, 2012, 4:20:02 AM2/14/12
to
On 2012-02-14, Raphael Hertzog <her...@debian.org> wrote:
> I wonder what's the proper way to handle this. In theory, it would be nice
> to deal with that at the dpkg-dev level but dpkg-dev is not at all
> involved in installing the changelog. And I believe that the bin-nmu
> process just adds a top-level entry to debian/changelog.
>
> So the code should go to dh_installchangelogs... but it doesn't seem to be
> a good idea to put the bin-nmu logic there in particular since we might
> extend it (see #440094).
>
> Somehow my suggestion is then to extend dpkg-parsechangelog to provide
> the required logic to split the changelog in its bin-nmu part and its
> usual content.
>
> dpkg-parsechangelog --split-binnmu <binnmu-part-file> <remaining-part-file>
>
> Then dh_installchangelogs could try to use this (and if it fails, fallback
> to the standard changelog installation).
>
> Does that sound sane? If yes, I can have a look at implementing this.
In theory sbuild could also offload this to dpkg-buildpackage by passing
> I wonder what's the proper way to handle this. In theory, it would be nice
> to deal with that at the dpkg-dev level but dpkg-dev is not at all
> involved in installing the changelog. And I believe that the bin-nmu
> process just adds a top-level entry to debian/changelog.
>
> So the code should go to dh_installchangelogs... but it doesn't seem to be
> a good idea to put the bin-nmu logic there in particular since we might
> extend it (see #440094).
>
> Somehow my suggestion is then to extend dpkg-parsechangelog to provide
> the required logic to split the changelog in its bin-nmu part and its
> usual content.
>
> dpkg-parsechangelog --split-binnmu <binnmu-part-file> <remaining-part-file>
>
> Then dh_installchangelogs could try to use this (and if it fails, fallback
> to the standard changelog installation).
>
> Does that sound sane? If yes, I can have a look at implementing this.
something like "--binnmu-version 2 --binnmu-changelog 'Rebuild for libfoo
transition'". The only thing that would be annoying is checking if the old
style or the new style must be used. (I.e. there must be some sort of feature
query first.)
Kind regards
Philipp Kern
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/slrnjjk9br...@kelgar.0x539.de
Niels Thykier
Feb 14, 2012, 6:50:04 AM2/14/12
to
On 2012-02-14 07:43, Russ Allbery wrote:
> [...]
I have no problem with Lintian accepting arch-qualified override files,
but I do not see the "strict" (i.e. "must") requirement[1]. Lintian
already allows you to do arch-specific overrides and 2.5.5 will even
allow architecture wildcards as well[2].
~Niels
[1] Exception being compressed override files, but of the 161 override
files on my system not a single one of them are compressed.
[2] http://lintian.debian.org/manual/section-2.4.html#section-2.4.1
Admittedly the link above does not have an example with it, but Lintian
(git) has an entire "section" dedicated to architecture specific
overrules. So there will be several examples with next release. :)
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
> [...]
>
> * Lintian should recognize arch-qualified override files, and multiarch:
> same packages must arch-qualify their override files. debhelper
> assistance is desired for this.
>
> [...]
> * Lintian should recognize arch-qualified override files, and multiarch:
> same packages must arch-qualify their override files. debhelper
> assistance is desired for this.
>
I have no problem with Lintian accepting arch-qualified override files,
but I do not see the "strict" (i.e. "must") requirement[1]. Lintian
already allows you to do arch-specific overrides and 2.5.5 will even
allow architecture wildcards as well[2].
~Niels
[1] Exception being compressed override files, but of the 161 override
files on my system not a single one of them are compressed.
[2] http://lintian.debian.org/manual/section-2.4.html#section-2.4.1
Admittedly the link above does not have an example with it, but Lintian
(git) has an entire "section" dedicated to architecture specific
overrules. So there will be several examples with next release. :)
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4F3A48...@thykier.net
Raphael Hertzog
Feb 14, 2012, 8:30:02 AM2/14/12
to
On Tue, 14 Feb 2012, Philipp Kern wrote:
> On 2012-02-14, Raphael Hertzog <her...@debian.org> wrote:
> > Somehow my suggestion is then to extend dpkg-parsechangelog to provide
> > the required logic to split the changelog in its bin-nmu part and its
> > usual content.
> >
> > dpkg-parsechangelog --split-binnmu <binnmu-part-file> <remaining-part-file>
> >
> > Then dh_installchangelogs could try to use this (and if it fails, fallback
> > to the standard changelog installation).
> >
> > Does that sound sane? If yes, I can have a look at implementing this.
>
> In theory sbuild could also offload this to dpkg-buildpackage by passing
> something like "--binnmu-version 2 --binnmu-changelog 'Rebuild for libfoo
> transition'". The only thing that would be annoying is checking if the old
> style or the new style must be used. (I.e. there must be some sort of feature
> query first.)
Yes but that doesn't change anything to the fact that dpkg-dev should not
> On 2012-02-14, Raphael Hertzog <her...@debian.org> wrote:
> > Somehow my suggestion is then to extend dpkg-parsechangelog to provide
> > the required logic to split the changelog in its bin-nmu part and its
> > usual content.
> >
> > dpkg-parsechangelog --split-binnmu <binnmu-part-file> <remaining-part-file>
> >
> > Then dh_installchangelogs could try to use this (and if it fails, fallback
> > to the standard changelog installation).
> >
> > Does that sound sane? If yes, I can have a look at implementing this.
>
> In theory sbuild could also offload this to dpkg-buildpackage by passing
> something like "--binnmu-version 2 --binnmu-changelog 'Rebuild for libfoo
> transition'". The only thing that would be annoying is checking if the old
> style or the new style must be used. (I.e. there must be some sort of feature
> query first.)
install files in the generated .deb. So we still need some interaction
with dh_installchangelogs... but your suggestion lead me to another
proposal.
dpkg-buildpackage --binary-version <ver> --binary-changelog 'foo'
could create debian/changelog.build with the given changelog version and
changelog entry.
dpkg-parsechangelog could be taught to read debian/changelog.build
before debian/changelog so that dpkg-parsechangelog continues to do the
right thing (when called from debian/rules).
And dh_installchangelogs can be taught to install debian/changelog.build
as /usr/share/doc/<foo>/changelog.Debian.build-$arch.
dpkg-buildpackage would clean up debian/changelog.build if it wasn't
passed the proper option. dpkg-source would learn to not include it in
generated source packages, too.
This looks like rather appealing to me. What do you think?
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214131...@rivendell.home.ouaza.com
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Guillem Jover
Feb 14, 2012, 9:10:01 AM2/14/12
to
On Mon, 2012-02-13 at 22:43:04 -0800, Russ Allbery wrote:
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
> * dpkg re-adds the refcounting implementation for multiarch, but along
> with a Policy requirement that packages that are multiarch must only
> contain files in classes 1 and 2 above.
>
> * All packages that want to be multiarch: same have to move all generated
> documentation into a separate package unless the maintainer has very
> carefully checked that the generated documentation will be byte-for-byte
> identical even across minor updates of the documentation generation
> tools and when run at different times.
If packages have to be split anyway to cope with the other cases, then
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
> * dpkg re-adds the refcounting implementation for multiarch, but along
> with a Policy requirement that packages that are multiarch must only
> contain files in classes 1 and 2 above.
>
> * All packages that want to be multiarch: same have to move all generated
> documentation into a separate package unless the maintainer has very
> carefully checked that the generated documentation will be byte-for-byte
> identical even across minor updates of the documentation generation
> tools and when run at different times.
the number of new packages which might not be needed otherwise will be
even smaller than the predicted amount, at which point it makes even
less sense to support refcnt'ing.
It also requires maintainers to carefully consider if the (doc, etc)
toolchains will generate predictible ouput.
Your proposal still requires papering over the other corner-cases.
> * Policy prohibits arch-varying data files in multiarch: same packages
> except in arch-qualified paths.
refcnt'ing or not.
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
arch-qualified pkgname solves this cleanly, and allows debhelper to
automatically deal with it. And for tools to just change where they
always look for those files in the M-A:same case regardless of the
package being binNMUed or not.
This still does not solve the other issues I listed, namely binNMUs
have to be performed in lock-step, more complicated transitions /
upgrades. And introduces different solutions for different problems,
while my proposal is generic for all cases.
So this is still pretty much unconvincing, and seems like clinging
into the refcnt'ing “solution” while it makes things overall more
complicated, will introduce inconsistency and incertainty to
maintainers, needs way more global changes to keep it going, etc.
What I'd change to my proposal in the summary mail, is that arch-indep
files might be considered for splitting at maintainers discretion,
when it actually seems worth it, in the same way we've handled
splitting arch-indep files from arch:any up to now. So for example a
couple of headers could be kept on the -dev package, or Ian's case on
essential and data files could also be kept on the same lib package,
as long as their paths are arch-qualified either trhough a pkgname:arch
or the multiarch triplet. This would reduce even more the amount of
newly split packages.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 14, 2012, 9:20:02 AM2/14/12
to
Russ Allbery writes ("Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
Thanks for this summary and analysis. I agree with your conclusions.
> Does this seem comprehensive to everyone? Am I missing any cases?
I think you have covered all of the cases that have been brought up on
this list, which I think are all of the important and frequent cases.
Thinking about other corner cases can be deferred for now because we
can put off converting affected packages until wheezy+1, or if we
really want to convert we can very probably add a "common" package.
So let us press on.
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
If we always put the binNMU changelog file in
/usr/share/doc/<package>/changelog.Debian.<package>:<arch>
then in the symlink case we can put it file in
/usr/share/doc/<symlink-target>/changelog.Debian.<original-package>:<arch>
and everything will work (apart from the fact that some minority of
changelog-reading tools will need to be taught to look at the new path).
> Please note that this is a bunch of work. I think the Lintian work is a
> good idea regardless, and it can start independently. I think the same is
> true of the binNMU changelog work, since this will address some
> long-standing issues with changelog handling in some situations, including
> resolving just how we're supposed to handle /usr/share/doc symlinks. But
> even with those aside, this is a lot of stuff that we need to agree on,
> and in some cases implement, in a fairly short timeline if this is going
> to make wheezy.
Yes. The work that absolutely needs to be done ASAP seems to be:
- put the refcounting back in dpkg
- lintian support for arch-qualified overrides
- update the binNMU machinery to write the new changelog file instead
Things that should be done but are not on the critical paths:
- transpose the restrictions on use of refcounting into policy
(for now they can go in a text file in dpkg-dev, or even just
a reference to your email)
- update changelog-reading tools to look for binNMU changelogs too
Things which we can do at our leisure:
- convert individual libraries
- think about whether to always arch-qualify the whole changelog
- think other refcounting corner cases (see my comments above)
> 5. Data files that vary by architecture. This includes big-endian
> vs. little-endian issues. These are simply incompatible with
> multiarch as currently designed, and incompatible with the obvious
> variations that I can think of, and will have to either be moved
> into arch-qualified directories (with corresponding patches to the
> paths from which the libraries load the data) or these packages
> can't be made multiarch.
Yes. Of these, arch-qualifying the path seem to be to be obviously
the right answer. Of course eg if the data files just come in big-
and little-endian, you can qualify the path with only the endianness
and use refcounting to allow the equal-endianness packages to share.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20282.26861....@chiark.greenend.org.uk
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
Yes.
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
> Does this seem comprehensive to everyone? Am I missing any cases?
this list, which I think are all of the important and frequent cases.
Thinking about other corner cases can be deferred for now because we
can put off converting affected packages until wheezy+1, or if we
really want to convert we can very probably add a "common" package.
So let us press on.
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
/usr/share/doc/<package>/changelog.Debian.<package>:<arch>
then in the symlink case we can put it file in
/usr/share/doc/<symlink-target>/changelog.Debian.<original-package>:<arch>
and everything will work (apart from the fact that some minority of
changelog-reading tools will need to be taught to look at the new path).
> Please note that this is a bunch of work. I think the Lintian work is a
> good idea regardless, and it can start independently. I think the same is
> true of the binNMU changelog work, since this will address some
> long-standing issues with changelog handling in some situations, including
> resolving just how we're supposed to handle /usr/share/doc symlinks. But
> even with those aside, this is a lot of stuff that we need to agree on,
> and in some cases implement, in a fairly short timeline if this is going
> to make wheezy.
- put the refcounting back in dpkg
- lintian support for arch-qualified overrides
- update the binNMU machinery to write the new changelog file instead
Things that should be done but are not on the critical paths:
- transpose the restrictions on use of refcounting into policy
(for now they can go in a text file in dpkg-dev, or even just
a reference to your email)
- update changelog-reading tools to look for binNMU changelogs too
Things which we can do at our leisure:
- convert individual libraries
- think about whether to always arch-qualify the whole changelog
- think other refcounting corner cases (see my comments above)
> 5. Data files that vary by architecture. This includes big-endian
> vs. little-endian issues. These are simply incompatible with
> multiarch as currently designed, and incompatible with the obvious
> variations that I can think of, and will have to either be moved
> into arch-qualified directories (with corresponding patches to the
> paths from which the libraries load the data) or these packages
> can't be made multiarch.
the right answer. Of course eg if the data files just come in big-
and little-endian, you can qualify the path with only the endianness
and use refcounting to allow the equal-endianness packages to share.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Josselin Mouette
Feb 14, 2012, 9:50:01 AM2/14/12
to
Le lundi 13 février 2012 à 22:43 -0800, Russ Allbery a écrit :
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
Thank you very much for your constructive work.
> There's been a lot of discussion of this, but it seems to have been fairly
> inconclusive. We need to decide what we're doing, if anything, for wheezy
> fairly soon, so I think we need to try to drive this discussion to some
> concrete conclusions.
> 3. Generated documentation. Here's where I think refcounting starts
> failing.
doxygen from -dev packages to -doc packages. But it really seems an
acceptable tradeoff between the amount of work required and the
cleanness of the solution.
> Does this seem comprehensive to everyone? Am I missing any cases?
architectures? Think of stuff like ld.so.conf, where some plugins or
library path is coded in a configuration file.
--
.''`. Josselin Mouette
: :' :
`. `'
`-
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 14, 2012, 9:50:03 AM2/14/12
to
Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
filename. But that would involve a lot of changes to
changelog-reading tools which we perhaps don't want to do right now.
Note that even if we decide to always arch-qualify, we will still have
lots of old packages so all changelog-reading tools will need to look
in both places.
For most changelog-reading tools it won't be very troublesome if they
accidentally don't spot a binNMU entry. So Russ's proposal is a good
step towards your proposal. And if we decide we don't need to go all
the way then it's good enough for now.
> This still does not solve the other issues I listed, namely binNMUs
> have to be performed in lock-step, more complicated transitions /
> upgrades.
I don't think I see where this is coming from. Are you talking about
variation in gzip output ? Given the evidence we've seen here, in
practice I think that is not going to be a problem. Certainly it
won't demand that binNMUs be performed in lock-step.
> So this is still pretty much unconvincing, and seems like clinging
> into the refcnt'ing “solution” while it makes things overall more
> complicated, will introduce inconsistency and incertainty to
> maintainers, needs way more global changes to keep it going, etc.
I think the refcounting approach is very worthwhile because it
eliminates unnecessary work (by human maintainers) in many simple
cases.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20282.28586....@chiark.greenend.org.uk
> On Mon, 2012-02-13 at 22:43:04 -0800, Russ Allbery wrote:
> > * The binNMU process is changed to add the binNMU changelog entry to an
> > arch-qualified file (changelog.Debian.arch, probably). We need to
> > figure out what this means if the package being binNMU'd has a
> > /usr/share/doc/<package> symlink to another package, though; it's not
> > obvious what to do here.
>
> > arch-qualified file (changelog.Debian.arch, probably). We need to
> > figure out what this means if the package being binNMU'd has a
> > /usr/share/doc/<package> symlink to another package, though; it's not
> > obvious what to do here.
>
> This requires IMO multitude of hacks when the simplest and obvious
> arch-qualified pkgname solves this cleanly, and allows debhelper to
> automatically deal with it. And for tools to just change where they
> always look for those files in the M-A:same case regardless of the
> package being binNMUed or not.
I agree that it would be nice to always arch-qualify the changelog
> arch-qualified pkgname solves this cleanly, and allows debhelper to
> automatically deal with it. And for tools to just change where they
> always look for those files in the M-A:same case regardless of the
> package being binNMUed or not.
filename. But that would involve a lot of changes to
changelog-reading tools which we perhaps don't want to do right now.
Note that even if we decide to always arch-qualify, we will still have
lots of old packages so all changelog-reading tools will need to look
in both places.
For most changelog-reading tools it won't be very troublesome if they
accidentally don't spot a binNMU entry. So Russ's proposal is a good
step towards your proposal. And if we decide we don't need to go all
the way then it's good enough for now.
> This still does not solve the other issues I listed, namely binNMUs
> have to be performed in lock-step, more complicated transitions /
> upgrades.
variation in gzip output ? Given the evidence we've seen here, in
practice I think that is not going to be a problem. Certainly it
won't demand that binNMUs be performed in lock-step.
> So this is still pretty much unconvincing, and seems like clinging
> into the refcnt'ing “solution” while it makes things overall more
> complicated, will introduce inconsistency and incertainty to
> maintainers, needs way more global changes to keep it going, etc.
eliminates unnecessary work (by human maintainers) in many simple
cases.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Jakub Wilk
Feb 14, 2012, 9:50:03 AM2/14/12
to
* Raphael Hertzog <her...@debian.org>, 2012-02-14, 14:17:
Are we sure than no existing package uses debian/changelog.build for
their own purposes?
Are we sure that all existing packages (and helpers) that parse
debian/changelog use dpkg-parsechangelog?
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012021414...@jwilk.net
>dpkg-buildpackage --binary-version <ver> --binary-changelog 'foo' could
>create debian/changelog.build with the given changelog version and
>changelog entry.
>
>dpkg-parsechangelog could be taught to read debian/changelog.build
>before debian/changelog so that dpkg-parsechangelog continues to do the
>right thing (when called from debian/rules).
>
>And dh_installchangelogs can be taught to install
>debian/changelog.build as
>/usr/share/doc/<foo>/changelog.Debian.build-$arch.
>
>dpkg-buildpackage would clean up debian/changelog.build if it wasn't
>passed the proper option. dpkg-source would learn to not include it in
>generated source packages, too.
>
>This looks like rather appealing to me. What do you think?
Yes, it does look appealing. But...
>create debian/changelog.build with the given changelog version and
>changelog entry.
>
>dpkg-parsechangelog could be taught to read debian/changelog.build
>before debian/changelog so that dpkg-parsechangelog continues to do the
>right thing (when called from debian/rules).
>
>And dh_installchangelogs can be taught to install
>debian/changelog.build as
>/usr/share/doc/<foo>/changelog.Debian.build-$arch.
>
>dpkg-buildpackage would clean up debian/changelog.build if it wasn't
>passed the proper option. dpkg-source would learn to not include it in
>generated source packages, too.
>
>This looks like rather appealing to me. What do you think?
Are we sure than no existing package uses debian/changelog.build for
their own purposes?
Are we sure that all existing packages (and helpers) that parse
debian/changelog use dpkg-parsechangelog?
--
Jakub Wilk
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Sven Joachim
Feb 14, 2012, 10:00:03 AM2/14/12
to
On 2012-02-14 15:28 +0100, Ian Jackson wrote:
> Guillem Jover writes:
>
>> This still does not solve the other issues I listed, namely binNMUs
>> have to be performed in lock-step, more complicated transitions /
>> upgrades.
>
> I don't think I see where this is coming from. Are you talking about
> variation in gzip output ? Given the evidence we've seen here, in
> practice I think that is not going to be a problem. Certainly it
> won't demand that binNMUs be performed in lock-step.
Guillem is referring to the need to to keep package versions in sync
across architectures, pretty much a necessity if you permit shared
files.
Cheers,
Sven
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/871upxc...@turtle.gmx.de
> Guillem Jover writes:
>
>> This still does not solve the other issues I listed, namely binNMUs
>> have to be performed in lock-step, more complicated transitions /
>> upgrades.
>
> I don't think I see where this is coming from. Are you talking about
> variation in gzip output ? Given the evidence we've seen here, in
> practice I think that is not going to be a problem. Certainly it
> won't demand that binNMUs be performed in lock-step.
across architectures, pretty much a necessity if you permit shared
files.
Cheers,
Sven
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Marvin Renich
Feb 14, 2012, 10:20:01 AM2/14/12
to
* Russ Allbery <r...@debian.org> [120214 01:48]:
diversions has merit and deserves further scrutiny. It essentially
implements refcnt-like behavior by using the existing diversion
mechanism. I did not see any message in the thread that even
acknowledged that part of his message. Did I miss something?
...Marvin
[1] http://lists.debian.org/debian-devel/2012/02/msg00511.html
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214151...@cleo.wdw
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
I thought Goswin's suggestion in [1] of having dpkg use implicit
> is a mix of the various solutions that have been discussed:
diversions has merit and deserves further scrutiny. It essentially
implements refcnt-like behavior by using the existing diversion
mechanism. I did not see any message in the thread that even
acknowledged that part of his message. Did I miss something?
...Marvin
[1] http://lists.debian.org/debian-devel/2012/02/msg00511.html
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 14, 2012, 10:20:02 AM2/14/12
to
Hi,
On Tue, 14 Feb 2012, Sven Joachim wrote:
> > Guillem Jover writes:
> >
> >> This still does not solve the other issues I listed, namely binNMUs
> >> have to be performed in lock-step, more complicated transitions /
> >> upgrades.
> >
> > I don't think I see where this is coming from. Are you talking about
> > variation in gzip output ? Given the evidence we've seen here, in
> > practice I think that is not going to be a problem. Certainly it
> > won't demand that binNMUs be performed in lock-step.
>
> Guillem is referring to the need to to keep package versions in sync
> across architectures, pretty much a necessity if you permit shared
> files.
It's a matter of defining "in sync" as "having the same source version"
instead of "having the same binary version".
But such a change will require updates for APT too.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
On Tue, 14 Feb 2012, Sven Joachim wrote:
> > Guillem Jover writes:
> >
> >> This still does not solve the other issues I listed, namely binNMUs
> >> have to be performed in lock-step, more complicated transitions /
> >> upgrades.
> >
> > I don't think I see where this is coming from. Are you talking about
> > variation in gzip output ? Given the evidence we've seen here, in
> > practice I think that is not going to be a problem. Certainly it
> > won't demand that binNMUs be performed in lock-step.
>
> Guillem is referring to the need to to keep package versions in sync
> across architectures, pretty much a necessity if you permit shared
> files.
instead of "having the same binary version".
But such a change will require updates for APT too.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214151...@rivendell.home.ouaza.com
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 14, 2012, 10:20:02 AM2/14/12
to
Hi,
On Tue, 14 Feb 2012, Guillem Jover wrote:
> > * All packages that want to be multiarch: same have to move all generated
> > documentation into a separate package unless the maintainer has very
> > carefully checked that the generated documentation will be byte-for-byte
> > identical even across minor updates of the documentation generation
> > tools and when run at different times.
>
> If packages have to be split anyway to cope with the other cases, then
> the number of new packages which might not be needed otherwise will be
> even smaller than the predicted amount, at which point it makes even
> less sense to support refcnt'ing.
Why are you so opposed to the refcnt'ing?
It's not such a big deal to maintain this feature in dpkg. And even if the
current implementation is not perfect, it can be improved later when dpkg
will store by itself checksums of provided files.
To me it looks like you don't like refcnt'ing and you're trying to find
some reasons to make it unacceptable.
> It also requires maintainers to carefully consider if the (doc, etc)
> toolchains will generate predictible ouput.
If the maintainer has to install files in non-standard path (because of
the need to arch-qualify it), it will also need maintainers to carefully
consider how to ensure that this move doesn't break anything.
It's not a white/black situation. You're trading one potential problem for
another. And the differing files are likely to be much more easy to spot
than other behaviour changes that might be implied by the move of some
files to arch qualified paths.
> Your proposal still requires papering over the other corner-cases.
Can you be explicit about which corner cases you're referring to ?
> This still does not solve the other issues I listed, namely binNMUs
> have to be performed in lock-step
Can you explain why? If the binnmu changelog is in a arch-specific file,
then we're free to bin-nmu packages separately.
dpkg must just ensure that all "M-A: same" packages have the same source
version (instead of the binary version as currently).
>, more complicated transitions / upgrades.
We have no experience on this. It's a bit early to say whether those
constraints are going to be problematic or not.
> And introduces different solutions for different problems, while my
> proposal is generic for all cases.
There's nothing like a generic solution. You still have to decide whether
you move files to a -common package or if you arch qualify them and keep
them in the M-A: same package. And in both cases, you have to evaluate the
implications, in terms of package installation ordering in one case, in
terms of modifications to do to properly support the arch-qualified files
in the other one.
While it may sound like "cleaner" from a theoretical point of view, I'm
not convinced that it's better than the approach outlined by Russ.
Also you completely ignore the fact that what you're proposing is an
important change for multi-arch packages that have already been converted
both in Debian and in Ubuntu. You're pushing back the work to package
maintainers when there's not reason to not deal with this at the build
infrastructure level.
To reduce some of the downsides associated to compressed files in M-A:
same packages, we could/should investigate how to not compress files
in such packages instead of duplicating them needlessly.
> So this is still pretty much unconvincing, and seems like clinging
> into the refcnt'ing “solution” while it makes things overall more
> complicated, will introduce inconsistency and incertainty to
> maintainers, needs way more global changes to keep it going, etc.
This is not a fair characterization of the situation. IMO "Global changes" are
better than "lots of maintainers having to do busy-work splitting their
packages".
You see inconsistency in Russ's proposal but you don't see
inconsistency/incertainty when you change the standard location of
changelog files.
And the "more complicated", it might be true at the dpkg level, but I
don't believe that it's true from the maintainers points of view.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
On Tue, 14 Feb 2012, Guillem Jover wrote:
> > * All packages that want to be multiarch: same have to move all generated
> > documentation into a separate package unless the maintainer has very
> > carefully checked that the generated documentation will be byte-for-byte
> > identical even across minor updates of the documentation generation
> > tools and when run at different times.
>
> If packages have to be split anyway to cope with the other cases, then
> the number of new packages which might not be needed otherwise will be
> even smaller than the predicted amount, at which point it makes even
> less sense to support refcnt'ing.
It's not such a big deal to maintain this feature in dpkg. And even if the
current implementation is not perfect, it can be improved later when dpkg
will store by itself checksums of provided files.
To me it looks like you don't like refcnt'ing and you're trying to find
some reasons to make it unacceptable.
> It also requires maintainers to carefully consider if the (doc, etc)
> toolchains will generate predictible ouput.
the need to arch-qualify it), it will also need maintainers to carefully
consider how to ensure that this move doesn't break anything.
It's not a white/black situation. You're trading one potential problem for
another. And the differing files are likely to be much more easy to spot
than other behaviour changes that might be implied by the move of some
files to arch qualified paths.
> Your proposal still requires papering over the other corner-cases.
> This still does not solve the other issues I listed, namely binNMUs
> have to be performed in lock-step
then we're free to bin-nmu packages separately.
dpkg must just ensure that all "M-A: same" packages have the same source
version (instead of the binary version as currently).
>, more complicated transitions / upgrades.
constraints are going to be problematic or not.
> And introduces different solutions for different problems, while my
> proposal is generic for all cases.
you move files to a -common package or if you arch qualify them and keep
them in the M-A: same package. And in both cases, you have to evaluate the
implications, in terms of package installation ordering in one case, in
terms of modifications to do to properly support the arch-qualified files
in the other one.
While it may sound like "cleaner" from a theoretical point of view, I'm
not convinced that it's better than the approach outlined by Russ.
Also you completely ignore the fact that what you're proposing is an
important change for multi-arch packages that have already been converted
both in Debian and in Ubuntu. You're pushing back the work to package
maintainers when there's not reason to not deal with this at the build
infrastructure level.
To reduce some of the downsides associated to compressed files in M-A:
same packages, we could/should investigate how to not compress files
in such packages instead of duplicating them needlessly.
> So this is still pretty much unconvincing, and seems like clinging
> into the refcnt'ing “solution” while it makes things overall more
> complicated, will introduce inconsistency and incertainty to
> maintainers, needs way more global changes to keep it going, etc.
better than "lots of maintainers having to do busy-work splitting their
packages".
You see inconsistency in Russ's proposal but you don't see
inconsistency/incertainty when you change the standard location of
changelog files.
And the "more complicated", it might be true at the dpkg level, but I
don't believe that it's true from the maintainers points of view.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214151...@rivendell.home.ouaza.com
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 14, 2012, 10:30:02 AM2/14/12
to
On Tue, 14 Feb 2012, Jakub Wilk wrote:
> Are we sure than no existing package uses debian/changelog.build for
> their own purposes?
No, but with debian/changelog.dpkg-build we should be safe.
> Are we sure than no existing package uses debian/changelog.build for
> their own purposes?
> Are we sure that all existing packages (and helpers) that parse
> debian/changelog use dpkg-parsechangelog?
relatively quickly (we could even do a full rebuild to try to verify
pro-actively).
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214152...@rivendell.home.ouaza.com
Raphael Hertzog
Feb 14, 2012, 10:40:02 AM2/14/12
to
On Tue, 14 Feb 2012, Marvin Renich wrote:
> I thought Goswin's suggestion in [1] of having dpkg use implicit
> diversions has merit and deserves further scrutiny.
I don't. diversions support 2 packages, the "diverted" one and the
> I thought Goswin's suggestion in [1] of having dpkg use implicit
> diversions has merit and deserves further scrutiny.
"diverting" one. Multi-Arch: same must support co-installation of
any number of packages.
So you can't reuse the existing logic without heavy modifications.
And changing the destination path at installation time is not a
good idea. What if the package really requires that specific version
of the file at the indicated path ? We will have configured the package
as if everything went fine when in fact it did not.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120214153...@rivendell.home.ouaza.com
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Guillem Jover
Feb 14, 2012, 11:50:01 AM2/14/12
to
On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > On Mon, 2012-02-13 at 22:43:04 -0800, Russ Allbery wrote:
> > > * The binNMU process is changed to add the binNMU changelog entry to an
> > > arch-qualified file (changelog.Debian.arch, probably). We need to
> > > figure out what this means if the package being binNMU'd has a
> > > /usr/share/doc/<package> symlink to another package, though; it's not
> > > obvious what to do here.
> >
> > This requires IMO multitude of hacks when the simplest and obvious
> > arch-qualified pkgname solves this cleanly, and allows debhelper to
> > automatically deal with it. And for tools to just change where they
> > always look for those files in the M-A:same case regardless of the
> > package being binNMUed or not.
>
> I agree that it would be nice to always arch-qualify the changelog
> filename. But that would involve a lot of changes to
> changelog-reading tools which we perhaps don't want to do right now.
I've never proposed to arch-qualify the filename for the stuff under
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > On Mon, 2012-02-13 at 22:43:04 -0800, Russ Allbery wrote:
> > > * The binNMU process is changed to add the binNMU changelog entry to an
> > > arch-qualified file (changelog.Debian.arch, probably). We need to
> > > figure out what this means if the package being binNMU'd has a
> > > /usr/share/doc/<package> symlink to another package, though; it's not
> > > obvious what to do here.
> >
> > This requires IMO multitude of hacks when the simplest and obvious
> > arch-qualified pkgname solves this cleanly, and allows debhelper to
> > automatically deal with it. And for tools to just change where they
> > always look for those files in the M-A:same case regardless of the
> > package being binNMUed or not.
>
> I agree that it would be nice to always arch-qualify the changelog
> filename. But that would involve a lot of changes to
> changelog-reading tools which we perhaps don't want to do right now.
/usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
which are the only ones needing the disambiguation. This is how dpkg
handles pkgname output, or how it stores their data in the db too.
And it should be easy to ask a multiarch enabled dpkg-query for example
to normalize the pkgname output to be used on those paths, or otherwise
do it by hand:
if M-A == same
pkgname:arch
else
pkgname
> Note that even if we decide to always arch-qualify, we will still have
> lots of old packages so all changelog-reading tools will need to look
> in both places.
> For most changelog-reading tools it won't be very troublesome if they
> accidentally don't spot a binNMU entry. So Russ's proposal is a good
> step towards your proposal. And if we decide we don't need to go all
> the way then it's good enough for now.
file anyway? I only know of packages.d.o. Any other tool reading from
the installed path, cannot really rely on it being present at all
anyway, per policy.
And in addition, binNMU split changelogs are going to be there forever,
and as such their possible double locations. While the possible double
location for M-A:same packages using pkgname:arch qualified pathnames
would only be temporary and disappear once the packages have been rebuilt
with a new debhelper which automatically installs them in the correct
place.
> > So this is still pretty much unconvincing, and seems like clinging
> > into the refcnt'ing “solution” while it makes things overall more
> > complicated, will introduce inconsistency and incertainty to
> > maintainers, needs way more global changes to keep it going, etc.
>
> I think the refcounting approach is very worthwhile because it
> eliminates unnecessary work (by human maintainers) in many simple
> cases.
splitting that would otherwise not be needed should be even less than
before (which was predicted at around 700), also as I mentioned there
too, nothing prevents us from arch-qualifying paths (with Debian arch
or multiarch triplet depending on the case) if that's more convenient
or safer (as per your essential data example), and is what we've been
doing anyway for arch-indep data shipped in arch:any packages all along.
Given the amount of hacks or special casing piling up to make refcnt'ing
workable, when all that's really needed is a one time handling (or a
possible additional change for already converted packages, for things
that debhelper might not be able to handle) of moving qualifying paths
or splitting into new packages, it really does not seem worth it, no.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 14, 2012, 1:40:02 PM2/14/12
to
Niels Thykier <ni...@thykier.net> writes:
> On 2012-02-14 07:43, Russ Allbery wrote:
>> [...]
>>
>> * Lintian should recognize arch-qualified override files, and multiarch:
>> same packages must arch-qualify their override files. debhelper
>> assistance is desired for this.
>>
>> [...]
> I have no problem with Lintian accepting arch-qualified override files,
> but I do not see the "strict" (i.e. "must") requirement[1]. Lintian
> already allows you to do arch-specific overrides and 2.5.5 will even
> allow architecture wildcards as well[2].
Ah, yes, you're right, that's a good point. If you use architecture
> On 2012-02-14 07:43, Russ Allbery wrote:
>> [...]
>>
>> * Lintian should recognize arch-qualified override files, and multiarch:
>> same packages must arch-qualify their override files. debhelper
>> assistance is desired for this.
>>
>> [...]
> I have no problem with Lintian accepting arch-qualified override files,
> but I do not see the "strict" (i.e. "must") requirement[1]. Lintian
> already allows you to do arch-specific overrides and 2.5.5 will even
> allow architecture wildcards as well[2].
restrictions in the overrides, then you can install the same override file
on all architectures, so this doesn't need to be dealt with immediately
(if at all; we only have to do this if we want to support installing
different override files per architecture, which isn't strictly
necessary).
> [1] Exception being compressed override files, but of the 161 override
> files on my system not a single one of them are compressed.
override files. In fact, I didn't know it could. :)
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/878vk58...@windlord.stanford.edu
Guillem Jover
Feb 14, 2012, 8:20:02 PM2/14/12
to
On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> I think the refcounting approach is very worthwhile because it
> eliminates unnecessary work (by human maintainers) in many simple
> cases.
Aside from what I said on my other reply, I just wanted to note that
> eliminates unnecessary work (by human maintainers) in many simple
> cases.
this seems to be a recurring point of tension in the project when it
comes to archive wide source package changes, where supposed short
term convenience (with its usually long term harmful effects) appears
to initially seduce people over what seems to be the cleaner although
slightly a bit more laborious solution.
Other recent-ish incarnations of this tension could be the build-arch
build-indep targets, or the build flag settings; where the former got
recently resolved so that the right thing to do is for *all* packages
needing to eventually support those targets, or for the latter which
got switched from the seemingly more convenient to the more laborious
but correct solution, that is, *all* packages need to set those build
flags by themselves.
This is a fundamental issue with how our source packages are handled,
and the freedom and power it gives to experiment and implement them
whatever way the maintainer wants, has the price that doing some
archive wide changes is sometimes more costly, than changing something
centrally and be done with it. But trying to workaround this by coming
up with stacks of hacked up solutions will not solve that fundamental
issue, and this kind of tension will keep coming up again and again,
as long as the foundation is not reworked. Either that, or the project
needs to accept that fact and learn to live with this kind of changes,
with patience.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Joey Hess
Feb 14, 2012, 11:00:01 PM2/14/12
to
Guillem Jover wrote:
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
>
> Other recent-ish incarnations of this tension could be the build-arch
> build-indep targets, or the build flag settings; where the former got
> recently resolved so that the right thing to do is for *all* packages
> needing to eventually support those targets, or for the latter which
> got switched from the seemingly more convenient to the more laborious
> but correct solution, that is, *all* packages need to set those build
> flags by themselves.
>
> This is a fundamental issue with how our source packages are handled,
> and the freedom and power it gives to experiment and implement them
> whatever way the maintainer wants, has the price that doing some
> archive wide changes is sometimes more costly, than changing something
> centrally and be done with it. But trying to workaround this by coming
> up with stacks of hacked up solutions will not solve that fundamental
> issue, and this kind of tension will keep coming up again and again,
> as long as the foundation is not reworked. Either that, or the project
> needs to accept that fact and learn to live with this kind of changes,
> with patience.
Very interesting mail. While I certianly agree with your examples, it's
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
>
> Other recent-ish incarnations of this tension could be the build-arch
> build-indep targets, or the build flag settings; where the former got
> recently resolved so that the right thing to do is for *all* packages
> needing to eventually support those targets, or for the latter which
> got switched from the seemingly more convenient to the more laborious
> but correct solution, that is, *all* packages need to set those build
> flags by themselves.
>
> This is a fundamental issue with how our source packages are handled,
> and the freedom and power it gives to experiment and implement them
> whatever way the maintainer wants, has the price that doing some
> archive wide changes is sometimes more costly, than changing something
> centrally and be done with it. But trying to workaround this by coming
> up with stacks of hacked up solutions will not solve that fundamental
> issue, and this kind of tension will keep coming up again and again,
> as long as the foundation is not reworked. Either that, or the project
> needs to accept that fact and learn to live with this kind of changes,
> with patience.
worth remembering the counterexample of the /usr/doc transition which
took approximately 5 years to complete[1], and probably could have been
accomplished quickly and without pain with a simple hack to dpkg.
Anyway, my worry about the refcounting approach (or perhaps M-A: same in
general) is not the details of the implementation in dpkg, but the added
mental complexity of dpkg now being able to have multiple distinct
packages installed under the same name. I had a brief exposure to rpm,
which can install multiple versions of the same package, and that was
the main cause of much confusing behavior in rpm. While dpkg's invariant
that all co-installable package names be unique (and have unique files)
has certianly led to lots of ugly package names, it's kept the users'
and developers' mental models quite simple.
I worry that we have barely begun to scratch the surface of the added
complexity of losing this invariant.
--
see shy jo
[1] To the extent it was ever completed.. master.debian.org still has
a vestigial /usr/doc/
Russ Allbery
Feb 14, 2012, 11:10:01 PM2/14/12
to
Joey Hess <jo...@debian.org> writes:
> Anyway, my worry about the refcounting approach (or perhaps M-A: same in
> general) is not the details of the implementation in dpkg, but the added
> mental complexity of dpkg now being able to have multiple distinct
> packages installed under the same name. I had a brief exposure to rpm,
> which can install multiple versions of the same package, and that was
> the main cause of much confusing behavior in rpm. While dpkg's invariant
> that all co-installable package names be unique (and have unique files)
> has certianly led to lots of ugly package names, it's kept the users'
> and developers' mental models quite simple.
> I worry that we have barely begun to scratch the surface of the added
> complexity of losing this invariant.
This does seem to be more M-A: same in general, to me, since whether we
> Anyway, my worry about the refcounting approach (or perhaps M-A: same in
> general) is not the details of the implementation in dpkg, but the added
> mental complexity of dpkg now being able to have multiple distinct
> packages installed under the same name. I had a brief exposure to rpm,
> which can install multiple versions of the same package, and that was
> the main cause of much confusing behavior in rpm. While dpkg's invariant
> that all co-installable package names be unique (and have unique files)
> has certianly led to lots of ugly package names, it's kept the users'
> and developers' mental models quite simple.
> I worry that we have barely begun to scratch the surface of the added
> complexity of losing this invariant.
have file overlaps or not we still have multiple packages with the same
name. Which will force changes in everything that deals with packages,
like Puppet, to be able to specify packages with particular architectures.
I definitely agree on the complexity this adds. But I don't think there's
an alternative to that complexity without using something like --sysroot
or mini-chroots, and I don't think those are satisfying solutions to the
set of problems we're trying to solve.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87pqdg4...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 15, 2012, 2:50:01 AM2/15/12
to
On Tue, 14 Feb 2012, Guillem Jover wrote:
> I've never proposed to arch-qualify the filename for the stuff under
> /usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
> the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
> which are the only ones needing the disambiguation. This is how dpkg
> handles pkgname output, or how it stores their data in the db too.
[...]
> I've never proposed to arch-qualify the filename for the stuff under
> /usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
> the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
> which are the only ones needing the disambiguation. This is how dpkg
> handles pkgname output, or how it stores their data in the db too.
> How many tools are there that actually read the binary package changelog
> file anyway?
There's apt-listchanges surely. And probably a bunch of other that are
> file anyway?
less known.
I don't know if it's worth it, but if we go down that route, and if we
want to keep /usr/share/doc/pkgname on user's systems we could create
a new command in dpkg-maintscript-helper to manage that path as
a symlink to the native "M-A: same" package (if possible, otherwise
to any installed arch). That dpkg-maintscript-helper call could be
auto-enabled by debhelper for "M-A: same" packages.
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120215074...@rivendell.home.ouaza.com
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 15, 2012, 9:40:01 AM2/15/12
to
Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> Russ Allbery writes ("Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> Russ Allbery writes ("Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
>> 5. Data files that vary by architecture. This includes big-endian
>> vs. little-endian issues. These are simply incompatible with
>> multiarch as currently designed, and incompatible with the obvious
>> variations that I can think of, and will have to either be moved
>> into arch-qualified directories (with corresponding patches to the
>> paths from which the libraries load the data) or these packages
>> can't be made multiarch.
>
> Yes. Of these, arch-qualifying the path seem to be to be obviously
> the right answer. Of course eg if the data files just come in big-
> and little-endian, you can qualify the path with only the endianness
> and use refcounting to allow the equal-endianness packages to share.
>
> Ian.
Preferably -data-be:all and -data-le:all packages if they can be build
>> vs. little-endian issues. These are simply incompatible with
>> multiarch as currently designed, and incompatible with the obvious
>> variations that I can think of, and will have to either be moved
>> into arch-qualified directories (with corresponding patches to the
>> paths from which the libraries load the data) or these packages
>> can't be made multiarch.
>
> Yes. Of these, arch-qualifying the path seem to be to be obviously
> the right answer. Of course eg if the data files just come in big-
> and little-endian, you can qualify the path with only the endianness
> and use refcounting to allow the equal-endianness packages to share.
>
> Ian.
irespective of the buildds endianness.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 15, 2012, 10:00:04 AM2/15/12
to
Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
>> This still does not solve the other issues I listed, namely binNMUs
>> have to be performed in lock-step, more complicated transitions /
>> upgrades.
>
> I don't think I see where this is coming from. Are you talking about
> variation in gzip output ? Given the evidence we've seen here, in
> practice I think that is not going to be a problem. Certainly it
> won't demand that binNMUs be performed in lock-step.
Note that splitting files (specifically changelog) into -common package
>> have to be performed in lock-step, more complicated transitions /
>> upgrades.
>
> I don't think I see where this is coming from. Are you talking about
> variation in gzip output ? Given the evidence we've seen here, in
> practice I think that is not going to be a problem. Certainly it
> won't demand that binNMUs be performed in lock-step.
would require an explicit versioned dependency on the -common package and
produce the same (or similar) lock-step problem for upgrades and
binNMUs. Arch qualifying the files on the other hand would avoid that.
Splitting data files into -common packages will also often need a close
versioned dependency forcing a lock-step of packages. But probably not
so terse that binNMUs would have to be lock-steped.
Overall I think the lock-step being required for reference counted files
won't have such a large effect as you might think.
I think the idea of splitting the binNMU changelog into an extra file is
a great idea as that would allow putting the changelog into -common:all
and depend on the source version and then have the binNMU changelog in
the foo:any package in the symlinked directory. For this to work the
binNMU changelog should be arch and pkg qualified, e.g.
/usr/share/doc/foo-common/Changelog
/usr/share/doc/foo-common/Changelog.binNMU-foo-amd64
/usr/share/doc/foo-common/Changelog.binNMU-bar-i386
/usr/share/doc/foo -> foo-common
/usr/share/doc/bar -> foo-common
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 15, 2012, 10:00:04 AM2/15/12
to
Russ Allbery <r...@debian.org> writes:
> Joey Hess <jo...@debian.org> writes:
>
>> Anyway, my worry about the refcounting approach (or perhaps M-A: same in
>> general) is not the details of the implementation in dpkg, but the added
>> mental complexity of dpkg now being able to have multiple distinct
>> packages installed under the same name. I had a brief exposure to rpm,
>> which can install multiple versions of the same package, and that was
>> the main cause of much confusing behavior in rpm. While dpkg's invariant
>> that all co-installable package names be unique (and have unique files)
>> has certianly led to lots of ugly package names, it's kept the users'
>> and developers' mental models quite simple.
>
>> I worry that we have barely begun to scratch the surface of the added
>> complexity of losing this invariant.
>
> This does seem to be more M-A: same in general, to me, since whether we
> have file overlaps or not we still have multiple packages with the same
> name. Which will force changes in everything that deals with packages,
> like Puppet, to be able to specify packages with particular architectures.
>
> I definitely agree on the complexity this adds. But I don't think there's
> an alternative to that complexity without using something like --sysroot
> or mini-chroots, and I don't think those are satisfying solutions to the
> set of problems we're trying to solve.
pkg:arch will still be unique and the dpkg/apt output will use the
> Joey Hess <jo...@debian.org> writes:
>
>> Anyway, my worry about the refcounting approach (or perhaps M-A: same in
>> general) is not the details of the implementation in dpkg, but the added
>> mental complexity of dpkg now being able to have multiple distinct
>> packages installed under the same name. I had a brief exposure to rpm,
>> which can install multiple versions of the same package, and that was
>> the main cause of much confusing behavior in rpm. While dpkg's invariant
>> that all co-installable package names be unique (and have unique files)
>> has certianly led to lots of ugly package names, it's kept the users'
>> and developers' mental models quite simple.
>
>> I worry that we have barely begun to scratch the surface of the added
>> complexity of losing this invariant.
>
> This does seem to be more M-A: same in general, to me, since whether we
> have file overlaps or not we still have multiple packages with the same
> name. Which will force changes in everything that deals with packages,
> like Puppet, to be able to specify packages with particular architectures.
>
> I definitely agree on the complexity this adds. But I don't think there's
> an alternative to that complexity without using something like --sysroot
> or mini-chroots, and I don't think those are satisfying solutions to the
> set of problems we're trying to solve.
architecture where required for uniqueness. So I think that after some
getting used to it it will be clear enough again.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Joey Hess
Feb 15, 2012, 11:30:02 AM2/15/12
to
Goswin von Brederlow wrote:
> pkg:arch will still be unique and the dpkg/apt output will use the
> architecture where required for uniqueness. So I think that after some
> getting used to it it will be clear enough again.
Here are a few examples of the problems I worry about. I have not
> pkg:arch will still be unique and the dpkg/apt output will use the
> architecture where required for uniqueness. So I think that after some
> getting used to it it will be clear enough again.
verified any of them, and they're clearly biased toward code I am
familiar with, which suggests there are many other similar problems.
* Puppet not only installs packages, it may remove them. A puppet config
that does dpkg --purge foo will fail if multiarch is enabled, now
it needs to find and remove foo:*
* dpkg-repack pkg:arch will create a package with that literal name (or fail)
* dpkg-reconfigure probably can't be used with M-A same packages.
debconf probably generally needs porting to multiarch.
* tasksel uses dpkg --query to work out if a task's dependencies are
installed. In the event that a library is directly part of a task,
this will fail when multiarch is enabled.
* Every piece of documentation that gives commands lines manipulating
library packages is potentially broken.
Seems like we need a release where multiarch is classed as an
experimental feature, which when enabled can break the system. But the
sort of problems above are the easy to anticipate ones; my real worry is
the unanticipated classes of problems. Especially if we find intractable
problems or levels of complexity introduced by dropping the unique
package name invariant.
My nightmare scenario is that we release with multiarch, discover that
it's a net negative for our users (proprietary software on amd64 aside,
nearly all the benefits are to developers AFAICS), and are stuck with it.
--
see shy jo
Ian Jackson
Feb 15, 2012, 12:00:03 PM2/15/12
to
Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> > I think the refcounting approach is very worthwhile because it
> > eliminates unnecessary work (by human maintainers) in many simple
> > cases.
>
> > eliminates unnecessary work (by human maintainers) in many simple
> > cases.
>
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
The refcnt doesn't just eliminate unnecessary multiarch
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
conversion work. It also eliminates unnecessary maintenance effort.
Maintaining a split package will be more work than without.
I think that over the lifetime of the multiarch deployment this extra
packaging work will far outweigh the extra maintenance and
documentation burden of the refcnt feature.
> [...] But trying to workaround this by coming
> up with stacks of hacked up solutions [...]
I disagree with your tendentious phrasing. The refcnt feature is not
a "hacked up solution" (nor a stack of them). It is entirely normal
in Debian core tools (as in any substantial piece of software serving
a lot of diverse needs) to have extra code to make it easier to deploy
or use in common cases simpler.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 15, 2012, 12:10:01 PM2/15/12
to
Joey Hess writes ("Re: Multiarch file overlap summary and proposal"):
> * dpkg-repack pkg:arch will create a package with that literal name (or fail)
This is of course also a (new) bug. There will be a lot of bugs like
this while we deploy multiarch.
> * dpkg-reconfigure probably can't be used with M-A same packages.
> debconf probably generally needs porting to multiarch.
I think the non-arch-qualified nature of debconf question ids is
probably a feature rather than a bug. These questions should be
arch-qualified by the package only if they need to be.
> [other examples]
Yes.
> Seems like we need a release where multiarch is classed as an
> experimental feature, which when enabled can break the system.
Certainly. That's what wheezy will be.
> But the sort of problems above are the easy to anticipate ones; my
> real worry is the unanticipated classes of problems. Especially if
> we find intractable problems or levels of complexity introduced by
> dropping the unique package name invariant.
I think the fact that (package name, architecture) is still unique on
the system will be sufficient to make it possible to update all
existing software to deal with the new model.
> My nightmare scenario is that we release with multiarch, discover that
> it's a net negative for our users (proprietary software on amd64 aside,
> nearly all the benefits are to developers AFAICS), and are stuck with it.
I don't think this is likely.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20283.57768....@chiark.greenend.org.uk
> Goswin von Brederlow wrote:
> > pkg:arch will still be unique and the dpkg/apt output will use the
> > architecture where required for uniqueness. So I think that after some
> > getting used to it it will be clear enough again.
>
> > pkg:arch will still be unique and the dpkg/apt output will use the
> > architecture where required for uniqueness. So I think that after some
> > getting used to it it will be clear enough again.
>
> Here are a few examples of the problems I worry about. I have not
> verified any of them, and they're clearly biased toward code I am
> familiar with, which suggests there are many other similar problems.
>
> * Puppet not only installs packages, it may remove them. A puppet config
> that does dpkg --purge foo will fail if multiarch is enabled, now
> it needs to find and remove foo:*
Yes.
> verified any of them, and they're clearly biased toward code I am
> familiar with, which suggests there are many other similar problems.
>
> * Puppet not only installs packages, it may remove them. A puppet config
> that does dpkg --purge foo will fail if multiarch is enabled, now
> it needs to find and remove foo:*
> * dpkg-repack pkg:arch will create a package with that literal name (or fail)
this while we deploy multiarch.
> * dpkg-reconfigure probably can't be used with M-A same packages.
> debconf probably generally needs porting to multiarch.
probably a feature rather than a bug. These questions should be
arch-qualified by the package only if they need to be.
> [other examples]
Yes.
> Seems like we need a release where multiarch is classed as an
> experimental feature, which when enabled can break the system.
> But the sort of problems above are the easy to anticipate ones; my
> real worry is the unanticipated classes of problems. Especially if
> we find intractable problems or levels of complexity introduced by
> dropping the unique package name invariant.
the system will be sufficient to make it possible to update all
existing software to deal with the new model.
> My nightmare scenario is that we release with multiarch, discover that
> it's a net negative for our users (proprietary software on amd64 aside,
> nearly all the benefits are to developers AFAICS), and are stuck with it.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Ian Jackson
Feb 15, 2012, 12:10:02 PM2/15/12
to
Russ Allbery writes ("Re: Multiarch file overlap summary and proposal"):
every library, the same way we add (part of) the version number to the
package name.
I think the current multiarch proposal is a better idea than that.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20283.57580....@chiark.greenend.org.uk
> I definitely agree on the complexity this adds. But I don't think there's
> an alternative to that complexity without using something like --sysroot
> or mini-chroots, and I don't think those are satisfying solutions to the
> set of problems we're trying to solve.
In principle we could have added the arch name to the package name of
> an alternative to that complexity without using something like --sysroot
> or mini-chroots, and I don't think those are satisfying solutions to the
> set of problems we're trying to solve.
every library, the same way we add (part of) the version number to the
package name.
I think the current multiarch proposal is a better idea than that.
Ian.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 15, 2012, 6:40:02 PM2/15/12
to
Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
> Joey Hess writes ("Re: Multiarch file overlap summary and proposal"):
> Joey Hess writes ("Re: Multiarch file overlap summary and proposal"):
>> Here are a few examples of the problems I worry about. I have not
>> verified any of them, and they're clearly biased toward code I am
>> familiar with, which suggests there are many other similar problems.
>> * Puppet not only installs packages, it may remove them. A puppet config
>> that does dpkg --purge foo will fail if multiarch is enabled, now
>> it needs to find and remove foo:*
> Yes.
Actually, why would that be the behavior? Why would dpkg --purge foo not
>> verified any of them, and they're clearly biased toward code I am
>> familiar with, which suggests there are many other similar problems.
>> * Puppet not only installs packages, it may remove them. A puppet config
>> that does dpkg --purge foo will fail if multiarch is enabled, now
>> it needs to find and remove foo:*
> Yes.
just remove foo for all architectures for which it's installed, and
require that if you want to remove only a specific architecture you then
use the expanded syntax?
This comes back to something that I started thinking about after Guillem's
message, namely that I think the version lockstep is a feature, not a
limitation. A lot of the complexity from biarch with RPM comes from the
fact that the packages are (as I recall) independent and can be of
different versions. This sounds like a nice feature, but I think it's
actually a huge source of potential confusion and isn't as useful as it
may look.
I think it would be better to have a world in which all the architectures
of the foo package on the system have to have the same version, because
then you don't have to treat foo:i386 and foo:amd64 like they're separate
packages. The list of installed architectures is an *attribute* of the
package. A package is still either installed or not installed, but when
it's installed, it can be installed for one or more architectures. But if
it's installed for multiple architectures, those architectures are always
upgraded together and always remain consistent. That avoids all weirdness
of having a package work differently because the version varies depending
on the ABI, and it significantly simplifies the mental model behind the
package.
Note that this obviously requires that a binNMU not be considered a
different version of the package for this purpose. But I think that too
makes sense. A binNMU *isn't* a full new version of the package; it's a
new build of the same version. We've historically been a bit sloppy about
this distinction, but I think it's a real distinction and a meaningful
one.
I have not looked in detail at the current multiarch implementations and I
have no idea how closely the above matches what is actually being done,
but I would advocate for it, even to the degree of embedding understanding
of binNMU versioning in the package tools so that they know a binNMU is a
new version from the perspective of needing to upgrade the package but not
a new version from the perspective of version mismatches between arches.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87ehtv6...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 15, 2012, 7:40:02 PM2/15/12
to
Guillem Jover <gui...@debian.org> writes:
> If packages have to be split anyway to cope with the other cases, then
> the number of new packages which might not be needed otherwise will be
> even smaller than the predicted amount, at which point it makes even
> less sense to support refcnt'ing.
I don't think the package count is really the interesting metric here,
> If packages have to be split anyway to cope with the other cases, then
> the number of new packages which might not be needed otherwise will be
> even smaller than the predicted amount, at which point it makes even
> less sense to support refcnt'ing.
unless the number of introduced packages is very large (see below about
-dev packages). I'm more concerned with maintainer time and with
dependency complexity, and with the known problems that we introduce
whenever we take tightly-coupled files and separate them into independent
packages.
> It also requires maintainers to carefully consider if the (doc, etc)
> toolchains will generate predictible ouput.
having to do the work to use arch-qualified directories. But I think it's
worth having the tradeoff available as an option.
> This still does not solve the other issues I listed, namely binNMUs have
> to be performed in lock-step, more complicated transitions /
> upgrades. And introduces different solutions for different problems,
> while my proposal is generic for all cases.
as well), and I think that I addressed the cases that you pointed out in
the set of cases that I discussed in my message apart from the point about
version lockstep.
I just posted separately about version lockstep: I think this is a
feature, not a bug, in our multiarch implementation. I think this is the
direction we *should* go, because it reduces the overall complexity of the
system. Yes, that requires treating binNMUs as something different than a
sourceful version change, but I think that's a good idea *anyway*, even in
the absence of multiarch. It's more semantically accurate, and it would
address other lingering problems we've had with binNMUs, or at least
reduce them.
As for the benefits of refcounting, there are three places where I think
this has substantial benefit, so let me talk through them:
1. If you look at the list of files that Steve gave in multiarch: same
packages in Ubuntu, most of the cases that don't fall into the known
documentation and package metadata areas are a bunch of separate
special cases. They don't fall easily into a handful of cases. *But*,
they are mostly all files in either textual or consistent binary
formats that are installed directly from the package and are not going
to change in a binNMU. That means that refcounting provides a nice
simplification: there are a bunch of random additional files of various
different types that can all be handled the same way, without
additional packaging complexity. They can't all be arch-qualified in
the same way as easily, plus arch-qualifying files that absolutely
should not differ between architectures and where that difference would
be a bug (such as with PAM configuration) seems wrong.
They can also be split out into an arch: all package. But here I think
it's worth remembering that splitting tightly-coupled files into
separate packages has real drawbacks and is something we should avoid
doing if we can. There are places where the advantages to doing so are
overwhelming (-dev packages from shared libraries, for example), but we
should be sure we're in that case.
This is something that working on Lintian for a while really drove home
for me. People split binary packages with large data into an arch: any
and arch: all package (often because Lintian recommends it to save
archive space), and they do it wrong *all the time*. They get the
dependencies wrong, or don't think about what files belong in which
package, or accidentally put an arch-specific file in the data package.
I have a package that does this sort of split (gnubg), and from
personal experience know that it's not an easy thing to maintain.
We're not saving packaging complexity by asking people to do this
instead of refcounting, IMO.
Also, there are other drawbacks of splitting closely coupled files into
separate packages even apart from packaging complexity. For example,
it's common for people to move the man pages and desktop files for
binaries into that arch: all data package too, since hey they're
arch-independent and in /usr/share and that's easy. But this is
usually the wrong thing to do. Now you have created the possibility of
having desktop files installed for binaries that don't exist, you've
made it much harder for tools like Lintian to check that your man pages
and desktop files are consistent with the binaries, and you have to be
very careful about versioning dependencies. You also create a separate
package that's artificial from the user's perspective, may not get
removed when the main package is removed, and shows up in apt-cache
search and similar interfaces despite the fact that the user doesn't
care about it at all.
I really don't like package splitting as our answer to everything. At
the least, it definitely isn't an obviously clean and simple solution.
2. Include files are, by quantity, a pretty substantial percentage of the
files we're talking about here once we start making our -dev packages
multiarch aware. Most include files these days aren't arch-specific,
since facilities like int32_t have removed a lot of the need to put
architecture-specific information in headers. Refcounting handles them
quite cleanly. They don't change during binNMUs, and they aren't
compressed.
I think splitting all -dev packages into -dev and -include is a
non-starter. This is a bunch of extra packaging work, it adds a bunch
of noise to our package lists that interferes with user searches, it
really does create a substantial number of additional packages (one for
every -dev package in the archive to a first approximation if we
multiarch everything), and it creates new artificial user support
problems, such as users who install the -include package and not the
-dev package and then don't understand why they can't link with the
library.
So, what we're really talking about here, in the absence of
refcounting, is moving the entirety of /usr/include into
/usr/include/<triplet>. Now, this *does* work technically; compilers
will find the headers, and the behavior will be what we want. But
again this is ongoing packaging complexity to install the headers in a
place other than where the upstream build system is going to put them.
It's also surprising. We already broke a few packages just by moving
the arch-specific include files of some packages there, which is
unavoidable. I had to do more ugly things to the ugly openafs build
system to cope, for example. And we just had a really uncomfortable
conversation on the GCC mailing list (that amounted to "what are those
crazy Debian people doing?") because these moves broke building GCC
from source because header files are no longer where they are
everywhere else.
Doing this for arch-specific headers is already problematic but not
really avoidable if we're going to move forward. But that's a smaller
change than doing it for nearly all headers, and I think we're really
going to surprise our users in some unpleasant ways if we do this.
Yes, software should not assume it knows the system header search
paths, but the fact remains that software *does*, and fixing all of it
is going to be painful and not produce a lot of good will with
upstreams.
3. Package metadata in /usr/share/doc/<package>. Yes, we can avoid
refcounting here by using <package>:<arch> instead, and I understand
the appeal of simplicitly there. But the more I think about this, the
more I think this is not the right model for how our packages should
look and not the idea that we should be exposing to users.
I don't think we should be encouraging the idea that <package>:i386 and
<package>:amd64 are two completely distinct installed packages that
have no relationship to each other. Rather, I think we should
encourage users to think of a single installed package named <package>,
which can be installed for one or more architectures. Creating
separate package metadata directories is the right thing to do if we
expect the i386 and amd64 installed packages to be different, have
different copyrights, have different NEWS.Debian files, different
READMEs, different changelogs (binNMU aside, as mentioned above), and
so forth. But this is going down exactly that complexity path that
Joey is talking about, IMO.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/874nur6...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
David Kalnischkies
Feb 15, 2012, 10:20:01 PM2/15/12
to
On Thu, Feb 16, 2012 at 00:39, Russ Allbery <r...@debian.org> wrote:
> Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
>> Joey Hess writes ("Re: Multiarch file overlap summary and proposal"):
>
>>> Here are a few examples of the problems I worry about. I have not
>>> verified any of them, and they're clearly biased toward code I am
>>> familiar with, which suggests there are many other similar problems.
>
>>> * Puppet not only installs packages, it may remove them. A puppet config
>>> that does dpkg --purge foo will fail if multiarch is enabled, now
>>> it needs to find and remove foo:*
>
>> Yes.
>
> Actually, why would that be the behavior? Why would dpkg --purge foo not
> just remove foo for all architectures for which it's installed, and
> require that if you want to remove only a specific architecture you then
> use the expanded syntax?
We (as in APT team and dpkg team) had a lot of discussions about that,
> Ian Jackson <ijac...@chiark.greenend.org.uk> writes:
>> Joey Hess writes ("Re: Multiarch file overlap summary and proposal"):
>
>>> Here are a few examples of the problems I worry about. I have not
>>> verified any of them, and they're clearly biased toward code I am
>>> familiar with, which suggests there are many other similar problems.
>
>>> * Puppet not only installs packages, it may remove them. A puppet config
>>> that does dpkg --purge foo will fail if multiarch is enabled, now
>>> it needs to find and remove foo:*
>
>> Yes.
>
> Actually, why would that be the behavior? Why would dpkg --purge foo not
> just remove foo for all architectures for which it's installed, and
> require that if you want to remove only a specific architecture you then
> use the expanded syntax?
see for starters (there a properly more in between the 10 months…)
[0] http://lists.debian.org/debian-dpkg/2011/01/msg00046.html
[1] http://lists.debian.org/debian-dpkg/2011/12/msg00005.html
In short, i think the biggest counter is that it feels unintuitive to
install a library (in native arch) with e.g. "apt-get install libfoo"
while you have to be specific at removal to avoid nuking 'unrelated' packages
with "apt-get remove libfoo".
You can counter this with: Then be specific while installing, but this not
only breaks backward-compatibility (especially dist-upgrades), it also adds
either unnecessary work for single-arch users or it feels strange to accept
a command in singlearch while it fails in multiarch…
(and yes, i know that i have 'lost' the 'battle' in [1], so the handling is
different in APT vs dpkg unfortunately, but that's life it seems…)
Maybe it would help this kind of discussions if we would have a list of
interface changes in dpkg and how someone is supposed to use it in the
future in case this has changed - i personally lost track of that.
(In case someone wants to know this for APT: foo is interpreted as foo:native.
If arch of foo is not native, the packagename is display as foo:arch.
So at least in my eyes brain-death simple - and backward compatible.
[and no, foo:* is currently not supported, but its on the todo])
> I think it would be better to have a world in which all the architectures
> of the foo package on the system have to have the same version, because
> then you don't have to treat foo:i386 and foo:amd64 like they're separate
> packages. The list of installed architectures is an *attribute* of the
> package. A package is still either installed or not installed, but when
> it's installed, it can be installed for one or more architectures. But if
> it's installed for multiple architectures, those architectures are always
> upgraded together and always remain consistent. That avoids all weirdness
> of having a package work differently because the version varies depending
> on the ABI, and it significantly simplifies the mental model behind the
> package.
libfoo:i386 v1 and libfoo:amd64 v2 are not supposed to be co-exist on a
system and as you describe it: its a feature in my eyes, too.
If you want co-installable different versions, call the packages libfoo1 and
libfoo2…
stable/testing users will not have a problem with this version-lock anyway
and unstable users should be able to deal with temporary uninstallable
packages caused by packages not built for all archs yet.
Its not exactly a new type of problem for them anyway…
(btw: doesn't britney do these lock-step upgrades all day long…)
(Note though that e.g. APT is not able to handle installed architectures as an
'attribute'. It not only has to handle them as 'different' packages (and more
specific different versions) to keep backward-compatibility, also different
dependencies on different architectures would make it pretty messy in
practice. But double-think is a requirement for APT development anyway. ;) )
> Note that this obviously requires that a binNMU not be considered a
> different version of the package for this purpose. But I think that too
> makes sense. A binNMU *isn't* a full new version of the package; it's a
> new build of the same version. We've historically been a bit sloppy about
> this distinction, but I think it's a real distinction and a meaningful
> one.
the 'sync' happens on the exact binary version.
Somewhere else in this multiarch-discussion was hinted that we could
sync on the version in (optional) Source tag instead to allow binNMU.
It's a bit too late (in my timezone) for me to do serious predictions on
difficult-levels on changing this in APT but i guess its relatively easy.
(the only problem i see is that i don't have ${source:Version} available
currently in the version structure, but we haven't even tried pushing
apt's abibreak to sid specifically as i feared "last-minute" changes…)
The question just remains if it is a good idea…
Best regards
David Kalnischkies
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 16, 2012, 1:50:02 PM2/16/12
to
I was thinking more about this, and I was finally able to put a finger on
why I don't like package splitting as a solution.
We know from prior experience with splitting packages for large
arch-independent data that one of the more common mistakes that we'll make
is to move the wrong files: to put into the arch-independent package a
file that's actually arch-dependent.
Look at the failure mode when that happens with the sort of package that
we're talking about splitting out of m-a:same packages:
* The arch-independent package gets arch-dependent content that happens to
match the architecture of the maintainer's build machine, since that's
the only place the arch-independent package is built. The maintainer
will by definition not notice, since the content is right for their
system.
* The maintainer is probably using a popular system type (usually either
i386 or amd64), and everyone else on that system type will also not
notice, so the bug can be latent for some time.
* Systems with the wrong architecture will get data files that have the
wrong format or the wrong information. This is usually not a case that
the software is designed to detect, so the result is normally random
segfaults or similar sorts of major bugs. The failure case for header
files is *particularly* bad: C software will generally compile fine with
the wrong-sized data types and then, at *runtime*, happily pass the
wrong data into the library, resulting in random segfaults and possibly
even data corruption. This won't happen until runtime, so could go
undetected for long periods of time.
This is a particularly nasty failure mode due to how long it can stay
undetected and how much havoc it causes.
Now, compare to the failure mode with refcounting if the maintainer
doesn't realize that an arch-specific file can't be shared:
* Each arch-specific package will continue to get the appropriate files
for that architecture. Each package will still be usable and consistent
independently, so users who don't care about multiarch won't ever see a
problem.
* Users who want to co-install separate architectures will immediately
encounter a dpkg error saying that the files aren't consistent. This
means they won't be able to co-install the packages, but dpkg will
prevent any actual harm from happening. The user will then report a bug
and the maintainer will realize what happened and be able to find some
way to fix it.
* Even better, we can automatically detect this error case by scanning the
archive for architecture pairs that have non-matching overlapping files
and deal with it proactively.
The refcounting failure mode behavior is just completely superior here.
And this *is* a mistake that we're going to make frequently; we know that
from past experience with splitting packages. Note that this problem
often happens because, when the maintainer originally split the package,
there was nothing arch-specific in the file, but upstream made it
arch-specific later on and the maintainer didn't notice. (It's very easy
to miss.) This is particularly common with header files.
Note that arch-qualifying all of the files does not have the problems of
package splitting, but it's also a much more intrusive fix.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87pqdeo...@windlord.stanford.edu
why I don't like package splitting as a solution.
We know from prior experience with splitting packages for large
arch-independent data that one of the more common mistakes that we'll make
is to move the wrong files: to put into the arch-independent package a
file that's actually arch-dependent.
Look at the failure mode when that happens with the sort of package that
we're talking about splitting out of m-a:same packages:
* The arch-independent package gets arch-dependent content that happens to
match the architecture of the maintainer's build machine, since that's
the only place the arch-independent package is built. The maintainer
will by definition not notice, since the content is right for their
system.
* The maintainer is probably using a popular system type (usually either
i386 or amd64), and everyone else on that system type will also not
notice, so the bug can be latent for some time.
* Systems with the wrong architecture will get data files that have the
wrong format or the wrong information. This is usually not a case that
the software is designed to detect, so the result is normally random
segfaults or similar sorts of major bugs. The failure case for header
files is *particularly* bad: C software will generally compile fine with
the wrong-sized data types and then, at *runtime*, happily pass the
wrong data into the library, resulting in random segfaults and possibly
even data corruption. This won't happen until runtime, so could go
undetected for long periods of time.
This is a particularly nasty failure mode due to how long it can stay
undetected and how much havoc it causes.
Now, compare to the failure mode with refcounting if the maintainer
doesn't realize that an arch-specific file can't be shared:
* Each arch-specific package will continue to get the appropriate files
for that architecture. Each package will still be usable and consistent
independently, so users who don't care about multiarch won't ever see a
problem.
* Users who want to co-install separate architectures will immediately
encounter a dpkg error saying that the files aren't consistent. This
means they won't be able to co-install the packages, but dpkg will
prevent any actual harm from happening. The user will then report a bug
and the maintainer will realize what happened and be able to find some
way to fix it.
* Even better, we can automatically detect this error case by scanning the
archive for architecture pairs that have non-matching overlapping files
and deal with it proactively.
The refcounting failure mode behavior is just completely superior here.
And this *is* a mistake that we're going to make frequently; we know that
from past experience with splitting packages. Note that this problem
often happens because, when the maintainer originally split the package,
there was nothing arch-specific in the file, but upstream made it
arch-specific later on and the maintainer didn't notice. (It's very easy
to miss.) This is particularly common with header files.
Note that arch-qualifying all of the files does not have the problems of
package splitting, but it's also a much more intrusive fix.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Carsten Hey
Feb 16, 2012, 5:20:01 PM2/16/12
to
* David Kalnischkies [2012-02-16 03:59 +0100]:
> > Actually, why would that be the behavior? Why would dpkg --purge foo not
> > just remove foo for all architectures for which it's installed, and
> > require that if you want to remove only a specific architecture you then
> > use the expanded syntax?
>
> We (as in APT team and dpkg team) had a lot of discussions about that,
> see for starters (there a properly more in between the 10 months…)
> [0] http://lists.debian.org/debian-dpkg/2011/01/msg00046.html
> [1] http://lists.debian.org/debian-dpkg/2011/12/msg00005.html
>
> In short, i think the biggest counter is that it feels unintuitive to
> install a library (in native arch) with e.g. "apt-get install libfoo"
> while you have to be specific at removal to avoid nuking 'unrelated' packages
> with "apt-get remove libfoo".
I would expect this (especially if the package foo is not a library, but
I would also expect this for libraries):
* apt-get install foo tries to install foo:native if possible, if it is
not possible, it tries to install the package foo from an other
architecture but ask before proceeding (as if additional dependencies
are required to install a package).
* apt-get remove foo removes all installed foo packages (on all
architectures).
This summarises how apt without multi-arch handles this, the above would
make apt with multi-arch also behave so:
apt-get install foo
------------------->
foo is not installed foo is installed
apt-get remove foo
<-------------------
> > Note that this obviously requires that a binNMU not be considered a
> > different version of the package for this purpose. But I think that too
> > makes sense. A binNMU *isn't* a full new version of the package; it's a
> > new build of the same version. We've historically been a bit sloppy about
> > this distinction, but I think it's a real distinction and a meaningful
> > one.
>
> Mhh. The current spec just forbids binNMU for M-A:same packages -
> the 'sync' happens on the exact binary version.
> Somewhere else in this multiarch-discussion was hinted that we could
> sync on the version in (optional) Source tag instead to allow binNMU.
> It's a bit too late (in my timezone) for me to do serious predictions on
> difficult-levels on changing this in APT but i guess its relatively easy.
> (the only problem i see is that i don't have ${source:Version} available
> currently in the version structure, but we haven't even tried pushing
> apt's abibreak to sid specifically as i feared "last-minute" changes…)
I'm not sure if you meant this with "Source tag", anyway, the 'Packages'
files miss the source version too, but this could be added as optional
field that would be used if it differs from the 'Version:' field.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012021622...@furrball.stateful.de
> On Thu, Feb 16, 2012 at 00:39, Russ Allbery <r...@debian.org> wrote:
> >>> it needs to find and remove foo:*
foo:all (or foo:any) instead of foo:* would save the need to quote it.
> >>> it needs to find and remove foo:*
> > Actually, why would that be the behavior? Why would dpkg --purge foo not
> > just remove foo for all architectures for which it's installed, and
> > require that if you want to remove only a specific architecture you then
> > use the expanded syntax?
>
> We (as in APT team and dpkg team) had a lot of discussions about that,
> see for starters (there a properly more in between the 10 months…)
> [0] http://lists.debian.org/debian-dpkg/2011/01/msg00046.html
> [1] http://lists.debian.org/debian-dpkg/2011/12/msg00005.html
>
> In short, i think the biggest counter is that it feels unintuitive to
> install a library (in native arch) with e.g. "apt-get install libfoo"
> while you have to be specific at removal to avoid nuking 'unrelated' packages
> with "apt-get remove libfoo".
I would also expect this for libraries):
* apt-get install foo tries to install foo:native if possible, if it is
not possible, it tries to install the package foo from an other
architecture but ask before proceeding (as if additional dependencies
are required to install a package).
* apt-get remove foo removes all installed foo packages (on all
architectures).
This summarises how apt without multi-arch handles this, the above would
make apt with multi-arch also behave so:
apt-get install foo
------------------->
foo is not installed foo is installed
apt-get remove foo
<-------------------
> > Note that this obviously requires that a binNMU not be considered a
> > different version of the package for this purpose. But I think that too
> > makes sense. A binNMU *isn't* a full new version of the package; it's a
> > new build of the same version. We've historically been a bit sloppy about
> > this distinction, but I think it's a real distinction and a meaningful
> > one.
>
> Mhh. The current spec just forbids binNMU for M-A:same packages -
> the 'sync' happens on the exact binary version.
> Somewhere else in this multiarch-discussion was hinted that we could
> sync on the version in (optional) Source tag instead to allow binNMU.
> It's a bit too late (in my timezone) for me to do serious predictions on
> difficult-levels on changing this in APT but i guess its relatively easy.
> (the only problem i see is that i don't have ${source:Version} available
> currently in the version structure, but we haven't even tried pushing
> apt's abibreak to sid specifically as i feared "last-minute" changes…)
files miss the source version too, but this could be added as optional
field that would be used if it differs from the 'Version:' field.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Carsten Hey
Feb 16, 2012, 5:50:03 PM2/16/12
to
* Russ Allbery [2012-02-16 10:43 -0800]:
example documentation that contains it's build date.
One way to address this is to use a new dpkg control file (placed in
/var/lib/dpkg/info) that lists files that dpkg considers to be equal
even if they differ.
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012021622...@furrball.stateful.de
> * Users who want to co-install separate architectures will immediately
> encounter a dpkg error saying that the files aren't consistent. This
> means they won't be able to co-install the packages, but dpkg will
> prevent any actual harm from happening. The user will then report a bug
> and the maintainer will realize what happened and be able to find some
> way to fix it.
>
> * Even better, we can automatically detect this error case by scanning the
> archive for architecture pairs that have non-matching overlapping files
> and deal with it proactively.
There are still files that differ that do not need to be fixed, for
> encounter a dpkg error saying that the files aren't consistent. This
> means they won't be able to co-install the packages, but dpkg will
> prevent any actual harm from happening. The user will then report a bug
> and the maintainer will realize what happened and be able to find some
> way to fix it.
>
> * Even better, we can automatically detect this error case by scanning the
> archive for architecture pairs that have non-matching overlapping files
> and deal with it proactively.
example documentation that contains it's build date.
One way to address this is to use a new dpkg control file (placed in
/var/lib/dpkg/info) that lists files that dpkg considers to be equal
even if they differ.
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 16, 2012, 6:00:01 PM2/16/12
to
Carsten Hey <car...@debian.org> writes:
> * Russ Allbery [2012-02-16 10:43 -0800]:
>> * Users who want to co-install separate architectures will immediately
>> encounter a dpkg error saying that the files aren't consistent. This
>> means they won't be able to co-install the packages, but dpkg will
>> prevent any actual harm from happening. The user will then report a bug
>> and the maintainer will realize what happened and be able to find some
>> way to fix it.
>> * Even better, we can automatically detect this error case by scanning the
>> archive for architecture pairs that have non-matching overlapping files
>> and deal with it proactively.
> There are still files that differ that do not need to be fixed, for
> example documentation that contains it's build date.
Every file that differs has to be fixed in the current multi-arch plan.
> * Russ Allbery [2012-02-16 10:43 -0800]:
>> * Users who want to co-install separate architectures will immediately
>> encounter a dpkg error saying that the files aren't consistent. This
>> means they won't be able to co-install the packages, but dpkg will
>> prevent any actual harm from happening. The user will then report a bug
>> and the maintainer will realize what happened and be able to find some
>> way to fix it.
>> * Even better, we can automatically detect this error case by scanning the
>> archive for architecture pairs that have non-matching overlapping files
>> and deal with it proactively.
> There are still files that differ that do not need to be fixed, for
> example documentation that contains it's build date.
Documentation that contains its build date is going to need to be split
out into a separate -docs package.
I'm fine with splitting documentation; that has far fewer problems than
splitting other types of files, since documentation isn't tightly coupled
at a level that breaks software.
> One way to address this is to use a new dpkg control file (placed in
> /var/lib/dpkg/info) that lists files that dpkg considers to be equal
> even if they differ.
sort of inconsistency depending on what package is installed first.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87d39ei...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Carsten Hey
Feb 17, 2012, 3:30:02 AM2/17/12
to
* Russ Allbery [2012-02-16 14:55 -0800]:
just a few small man pages.
> I'm fine with splitting documentation; that has far fewer problems than
> splitting other types of files, since documentation isn't tightly coupled
> at a level that breaks software.
debianutils uses a special make target 'prebuild' in debian/rules to
update build system related files and PO files before the actual source
package is built.
This basic idea also could be used to build problematic documentation
files on the maintainers computer before he/she builds the package. The
other targets would then install the prebuilt documentation into the
package without the need to build it first. A proper dependency on
debian/$prebuilt_doc could ensure that maintainers do not forget to run
debian/rules prebuild.
If maintainers choose to use such a target, suggesting a common name for
it in the developers reference could be reasonable.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120217082...@furrball.stateful.de
> Carsten Hey <car...@debian.org> writes:
> > There are still files that differ that do not need to be fixed, for
> > example documentation that contains it's build date.
>
> Every file that differs has to be fixed in the current multi-arch plan.
> Documentation that contains its build date is going to need to be split
> out into a separate -docs package.
I doubt that ftpmaster would be happy about -doc packages that contain
> > There are still files that differ that do not need to be fixed, for
> > example documentation that contains it's build date.
>
> Every file that differs has to be fixed in the current multi-arch plan.
> Documentation that contains its build date is going to need to be split
> out into a separate -docs package.
just a few small man pages.
> I'm fine with splitting documentation; that has far fewer problems than
> splitting other types of files, since documentation isn't tightly coupled
> at a level that breaks software.
update build system related files and PO files before the actual source
package is built.
This basic idea also could be used to build problematic documentation
files on the maintainers computer before he/she builds the package. The
other targets would then install the prebuilt documentation into the
package without the need to build it first. A proper dependency on
debian/$prebuilt_doc could ensure that maintainers do not forget to run
debian/rules prebuild.
If maintainers choose to use such a target, suggesting a common name for
it in the developers reference could be reasonable.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 17, 2012, 3:50:01 AM2/17/12
to
>> Every file that differs has to be fixed in the current multi-arch plan.
>> Documentation that contains its build date is going to need to be split
>> out into a separate -docs package.
> I doubt that ftpmaster would be happy about -doc packages that contain
> just a few small man pages.
I think they'll be okay with it when it's the cost of reasonable
>> Documentation that contains its build date is going to need to be split
>> out into a separate -docs package.
> I doubt that ftpmaster would be happy about -doc packages that contain
> just a few small man pages.
multiarch.
I feel fairly strongly that it isn't sane to have a file overlap when the
file doesn't match. You then lose the error detection when there are real
problems, and I don't trust any of us, myself included, to correctly tag
files where it doesn't matter.
On this front, I agree with Guillem: some amount of package splitting is
fine, and package splitting instead of additional complexity, like tagging
files that are allowed to vary, looks like a good tradeoff to me. The
splitting that I'm worried about is where the files are tightly coupled,
which is not the case for development man pages that are now in -dev
packages.
> debianutils uses a special make target 'prebuild' in debian/rules to
> update build system related files and PO files before the actual source
> package is built.
> This basic idea also could be used to build problematic documentation
> files on the maintainers computer before he/she builds the package. The
> other targets would then install the prebuilt documentation into the
> package without the need to build it first. A proper dependency on
> debian/$prebuilt_doc could ensure that maintainers do not forget to run
> debian/rules prebuild.
> If maintainers choose to use such a target, suggesting a common name for
> it in the developers reference could be reasonable.
upstream (I build POD-generated man pages from my autogen script, and in
Debian packaging don't bother to regenerate them).
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87vcn5h...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
David Kalnischkies
Feb 17, 2012, 8:40:02 AM2/17/12
to
On Thu, Feb 16, 2012 at 23:10, Carsten Hey <car...@debian.org> wrote:
> * David Kalnischkies [2012-02-16 03:59 +0100]:
>> On Thu, Feb 16, 2012 at 00:39, Russ Allbery <r...@debian.org> wrote:
>> >>> it needs to find and remove foo:*
>
> foo:all (or foo:any) instead of foo:* would save the need to quote it.
:all is already an architecture and currently it seems like dpkg accepts
> * David Kalnischkies [2012-02-16 03:59 +0100]:
>> On Thu, Feb 16, 2012 at 00:39, Russ Allbery <r...@debian.org> wrote:
>> >>> it needs to find and remove foo:*
>
> foo:all (or foo:any) instead of foo:* would save the need to quote it.
only that while APT will accept :amd64 (or whatever is native), too,
partly for internal reasons, but also because the difference for an apt-get
user is not really important. Either way, overloading wouldn't be nice.
:any has a special meaning in build-dependencies already meaning - surprise
surprise - give me any package. Overloading would be uncool, beside that
its the wrong word for it anyway if you want all, and not just any package.
>> > Actually, why would that be the behavior? Why would dpkg --purge foo not
>> > just remove foo for all architectures for which it's installed, and
>> > require that if you want to remove only a specific architecture you then
>> > use the expanded syntax?
>>
>> We (as in APT team and dpkg team) had a lot of discussions about that,
>> see for starters (there a properly more in between the 10 months…)
>> [0] http://lists.debian.org/debian-dpkg/2011/01/msg00046.html
>> [1] http://lists.debian.org/debian-dpkg/2011/12/msg00005.html
>>
>> In short, i think the biggest counter is that it feels unintuitive to
>> install a library (in native arch) with e.g. "apt-get install libfoo"
>> while you have to be specific at removal to avoid nuking 'unrelated' packages
>> with "apt-get remove libfoo".
>
> I would expect this (especially if the package foo is not a library, but
> I would also expect this for libraries):
detect that the package foo is in fact a library and not, say, a
plugin, a dev-package, a dbg-package or a future-coinstallable binary.
And the foo:* default would be okay and intuitive for all of those?
You also skipped the part of backward-compatibility with tools which
expect a single line/stanza of response and commands like
dpkg --{get,set}-selection which by definition work only with a single
package.
And backward-compatibility means in this context also to support
a dist-upgrade from squeeze to wheezy. If a new version of dpkg
doesn't accept the 'old' commands APT uses the upgrade will
be a pain.
The two threads i mentioned contain a lot more of these
considerations, so it might be in order to read these before
coming up with 'new' ideas.
> * apt-get remove foo removes all installed foo packages (on all
> architectures).
apt-get pre-multi-arch handled only packages from native arch (and :all),
so if you had installed foreign packages with dpkg --force-architecture
apt-get would have ignored it, not removed it.
> This summarises how apt without multi-arch handles this, the above would
> make apt with multi-arch also behave so:
>
> apt-get install foo
> ------------------->
> foo is not installed foo is installed
> apt-get remove foo
> <-------------------
The current implementation of always foo == foo:native doesn't fail
your diagram, too, so what is this going to show us?
>> (the only problem i see is that i don't have ${source:Version} available
>> currently in the version structure, but we haven't even tried pushing
>> apt's abibreak to sid specifically as i feared "last-minute" changes…)
>
> I'm not sure if you meant this with "Source tag", anyway, the 'Packages'
> files miss the source version too, but this could be added as optional
> field that would be used if it differs from the 'Version:' field.
Package: 3depict
Source: 3depict (0.0.9-1)
Version: 0.0.9-1+b1
[…]
It's used in other places in APT, e.g. 'apt-get source', which just looks
at the Packages file stanza. That's fine as this isn't a speed critical
operation - but if we want it for the lock-step operation apt needs that
piece of information in its internal structures for fast access to it and
adding new fields in these structures will require an abibreak.
That's the intended meaning of the quoted sentence.
Best regards
David Kalnischkies
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Jonathan Nieder
Feb 17, 2012, 9:50:01 AM2/17/12
to
David Kalnischkies wrote:
> You generously left out the paragraph describing how APT should
> detect that the package foo is in fact a library and not, say, a
> plugin, a dev-package, a dbg-package or a future-coinstallable binary.
> And the foo:* default would be okay and intuitive for all of those?
Yes, the foo:native default for installation and foo:* default for
> You generously left out the paragraph describing how APT should
> detect that the package foo is in fact a library and not, say, a
> plugin, a dev-package, a dbg-package or a future-coinstallable binary.
> And the foo:* default would be okay and intuitive for all of those?
removal would be intuitive for all of those.
See [1] for a mental model.
Hope that helps,
Jonathan
[1] http://thread.gmane.org/gmane.linux.debian.devel.dpkg.general/14028/focus=14031
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120217144631.GA5039@burratino
David Kalnischkies
Feb 17, 2012, 11:40:02 AM2/17/12
to
On Fri, Feb 17, 2012 at 15:46, Jonathan Nieder <jrni...@gmail.com> wrote:
> David Kalnischkies wrote:
>
>> You generously left out the paragraph describing how APT should
>> detect that the package foo is in fact a library and not, say, a
>> plugin, a dev-package, a dbg-package or a future-coinstallable binary.
>> And the foo:* default would be okay and intuitive for all of those?
>
> Yes, the foo:native default for installation and foo:* default for
> removal would be intuitive for all of those.
>
> See [1] for a mental model.
>
> Hope that helps,
> Jonathan
>
> [1] http://thread.gmane.org/gmane.linux.debian.devel.dpkg.general/14028/focus=14031
Why would it be intuitive to add a specific value for the arch attribute with
> David Kalnischkies wrote:
>
>> You generously left out the paragraph describing how APT should
>> detect that the package foo is in fact a library and not, say, a
>> plugin, a dev-package, a dbg-package or a future-coinstallable binary.
>> And the foo:* default would be okay and intuitive for all of those?
>
> Yes, the foo:native default for installation and foo:* default for
> removal would be intuitive for all of those.
>
> See [1] for a mental model.
>
> Hope that helps,
> Jonathan
>
> [1] http://thread.gmane.org/gmane.linux.debian.devel.dpkg.general/14028/focus=14031
apt-get install foo # arch |= native
but remove all values of the attribute with
apt-get remove foo # arch &= ~all-architectures
?
Isn't it more intuitive to have it this way:
apt-get remove foo # arch &= ~native
?
Maybe we just have different definitions of intuitive.
Best regards
David Kalnischkies
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Jonathan Nieder
Feb 17, 2012, 12:10:02 PM2/17/12
to
David Kalnischkies wrote:
> Why would it be intuitive to add a specific value for the arch attribute with
> apt-get install foo # arch |= native
> but remove all values of the attribute with
> apt-get remove foo # arch &= ~all-architectures
> ?
>
> Isn't it more intuitive to have it this way:
> apt-get remove foo # arch &= ~native
> ?
>
> Maybe we just have different definitions of intuitive.
Intuitions vary from person to person; that's definitely not news.
> Why would it be intuitive to add a specific value for the arch attribute with
> apt-get install foo # arch |= native
> but remove all values of the attribute with
> apt-get remove foo # arch &= ~all-architectures
> ?
>
> Isn't it more intuitive to have it this way:
> apt-get remove foo # arch &= ~native
> ?
>
> Maybe we just have different definitions of intuitive.
But I really think this is something anyone can get used to. In the
examples you listed above:
apt-get install foo; # install foo with default arch-list (native)
apt-get remove foo; # remove foo
If foo is installed for no architectures, that does not mean it is
installed with an empty architecture list. It means it is simply not
installed.
In practice, that would match what I want to do, too.
* There is a web browser I would like to use. I don't care which
arch --- that's an implementation detail.
apt-get install iceweasel
* Oops, never mind --- not interested in using that web browser any
more.
apt-get --purge remove iceweasel
* I've never heard of this multiarch stuff, but the unpackaged
software I am trying to install is giving complaints about missing
libfoo.so.1
apt-get install libfoo1
* Ok, now I've learned about multiarch, and I want to install libfoo
to satisfy a dependency for a binary on a foreign architecture.
apt-get install libfoo1:amd64
* I don't want libfoo any more --- remove it completely from the
system.
apt-get --purge remove libfoo1
"Wait!" you might protest. "Isn't that last command too aggressive?"
After all, I did not specify which architecture motivated the removal
of libfoo1. Maybe I was removing libfoo1 for the sake of my
unpackaged i386 software but I still need it for unpackaged amd64
software, and apt could help me out by picking the architecture I
intended and not removing it elsewhere, right?
But no, that would not be helpful at all. It's true that libfoo1
might be installed for more than one reason and I might have forgotten
about some and therefore remove it when that is not warranted, but
that's true whether multiarch is involved or not. This safety feature
does not add any real consistent safety.
I can think of only one advantage to making "apt-get remove libfoo1"
remove libfoo1:native, though it's a good one. That's to support
muscle memory and scripts that rely on the "libfoo1 always means
libfoo1:native" semantics that have been present in Ubuntu for a
little while. I think it's worth losing that, since as we've seen,
most scripts dealing with cases like this are going to need changes to
work with "multiarch: same" packages anyway (and humans can grow to
appreciate the simple mental model Russ suggested).
Jonathan
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Jonathan Nieder
Feb 17, 2012, 12:20:03 PM2/17/12
to
Jonathan Nieder wrote:
> David Kalnischkies wrote:
>> Why would it be intuitive to add a specific value for the arch attribute with
>> apt-get install foo # arch |= native
>> but remove all values of the attribute with
>> apt-get remove foo # arch &= ~all-architectures
>> ?
[...]
> David Kalnischkies wrote:
>> Why would it be intuitive to add a specific value for the arch attribute with
>> apt-get install foo # arch |= native
>> but remove all values of the attribute with
>> apt-get remove foo # arch &= ~all-architectures
>> ?
> But I really think this is something anyone can get used to. In the
> examples you listed above:
>
> apt-get install foo; # install foo with default arch-list (native)
> apt-get remove foo; # remove foo
>
> If foo is installed for no architectures, that does not mean it is
> installed with an empty architecture list. It means it is simply not
> installed.
Ok, now I think I figured out the inconsistency you are pointing to.
> examples you listed above:
>
> apt-get install foo; # install foo with default arch-list (native)
> apt-get remove foo; # remove foo
>
> If foo is installed for no architectures, that does not mean it is
> installed with an empty architecture list. It means it is simply not
> installed.
If i386 is the native architecture, what would you expect the
following sequence of commands to do?
apt-get install linux-image-3.2.0-1-amd64:amd64
... wait a few weeks ...
apt-get install linux-image-3.2.0-1-amd64
I would expect it to install the kernel with 'Architecture: amd64' and
then to upgrade it.
So the proposed semantics are not quite 'arch |= native'. They are
more like 'arch defaults to native for non-installed packages'.
Jonathan
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Carsten Hey
Feb 17, 2012, 2:00:02 PM2/17/12
to
* David Kalnischkies [2012-02-17 14:15 +0100]:
My impression was that you think very library centric. All I wrote was
(in other words), that we should consider non-library packages as much
as library packages, and I did not write nor implied that libraries
should be handled in a different way.
> Is 'apt-get remove foo+' then going to install all foo's or just one?
"apt-get install g+++" is a weird syntax.
> The current implementation of always foo == foo:native doesn't fail
> your diagram, too, so what is this going to show us?
It depends on how one reads it, anyway, examples I consider to be
inconsistent are more helpful than a diagram without clear semantic.
# dpkg --print-architecture
amd64
# perl -00 -lne 'print if /^Package: (clang|tendra)$/m &&
/^Status: install ok installed$/m' /var/lib/dpkg/status \
| awk '/^Package:/ {printf "%s:", $2} /^Architecture:/ {print $2}'
tendra:i386
clang:i386
I was not able to find a command that shows this information.
# apt-cache policy tendra | sed -n 1p
tendra:i386:
# apt-cache policy clang | sed -n 1p
clang:
# apt-get remove tendra
The following packages will be REMOVED:
tendra:i386
# apt-get remove clang
Package clang is not installed, so not removed
The above shows that it seems to depend on the availability of
foo:native if apt-get remove foo removes foo:foreign.
# dpkg -l | awk '$2=="clang"{print}'
ii clang 3.0-5 Low-Level ...
# dpkg -S bin/clang
clang: /usr/bin/clang
clang: /usr/bin/clang++
# dpkg -r clang
dpkg: warning: there's no installed package matching clang
# apt-get remove clang
Package clang is not installed, so not removed
According to dpkg's command line interface, the file /usr/bin/clang is
in the package clang and dpkg -l shows it as installed, but it can not
be removed using this name, neither by apt nor by dpkg.
# dpkg -l | grep libzookeeper-st2
ii libzookeeper-st2:amd64 3.3.4+dfsg1-3 Single ...
ii libzookeeper-st2:i386 3.3.4+dfsg1-3 Single ...
Unlike the above dpkg -l output showing the foreign clang package,
libzookeeper-st2 is shown with the architecture appended.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120217185...@furrball.stateful.de
> You generously left out the paragraph describing how APT should
> detect that the package foo is in fact a library ...
My impression was that you think very library centric. All I wrote was
(in other words), that we should consider non-library packages as much
as library packages, and I did not write nor implied that libraries
should be handled in a different way.
> Is 'apt-get remove foo+' then going to install all foo's or just one?
> The current implementation of always foo == foo:native doesn't fail
> your diagram, too, so what is this going to show us?
inconsistent are more helpful than a diagram without clear semantic.
# dpkg --print-architecture
amd64
# perl -00 -lne 'print if /^Package: (clang|tendra)$/m &&
/^Status: install ok installed$/m' /var/lib/dpkg/status \
| awk '/^Package:/ {printf "%s:", $2} /^Architecture:/ {print $2}'
tendra:i386
clang:i386
I was not able to find a command that shows this information.
# apt-cache policy tendra | sed -n 1p
tendra:i386:
# apt-cache policy clang | sed -n 1p
clang:
# apt-get remove tendra
The following packages will be REMOVED:
tendra:i386
# apt-get remove clang
Package clang is not installed, so not removed
The above shows that it seems to depend on the availability of
foo:native if apt-get remove foo removes foo:foreign.
# dpkg -l | awk '$2=="clang"{print}'
ii clang 3.0-5 Low-Level ...
# dpkg -S bin/clang
clang: /usr/bin/clang
clang: /usr/bin/clang++
# dpkg -r clang
dpkg: warning: there's no installed package matching clang
# apt-get remove clang
Package clang is not installed, so not removed
According to dpkg's command line interface, the file /usr/bin/clang is
in the package clang and dpkg -l shows it as installed, but it can not
be removed using this name, neither by apt nor by dpkg.
# dpkg -l | grep libzookeeper-st2
ii libzookeeper-st2:amd64 3.3.4+dfsg1-3 Single ...
ii libzookeeper-st2:i386 3.3.4+dfsg1-3 Single ...
Unlike the above dpkg -l output showing the foreign clang package,
libzookeeper-st2 is shown with the architecture appended.
Regards
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Carsten Hey
Feb 17, 2012, 2:20:02 PM2/17/12
to
* David Kalnischkies [2012-02-17 17:20 +0100]:
Package: foo
Depends: a, b, c
Conflicts: x, y, z
To be able to install foo, you need to:
* ensure that "a && b && c" is true.
The command to do so is apt-get install a b c
* ensure that "x || y || z" is false.
The command to do so is (or rather "should in my opinion be")
apt-get remove x y z
To satisfy the dependency line, *all* packages in it need to be installed.
To "satisfy" a conflicts field (that is, there is a conflict), *any* of
the packages in it needs to be installed.
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
> Why would it be intuitive to add a specific value for the arch attribute with
> apt-get install foo # arch |= native
> but remove all values of the attribute with
> apt-get remove foo # arch &= ~all-architectures
> ?
We had a similar discussion years ago.
> apt-get install foo # arch |= native
> but remove all values of the attribute with
> apt-get remove foo # arch &= ~all-architectures
> ?
Package: foo
Depends: a, b, c
Conflicts: x, y, z
To be able to install foo, you need to:
* ensure that "a && b && c" is true.
The command to do so is apt-get install a b c
* ensure that "x || y || z" is false.
The command to do so is (or rather "should in my opinion be")
apt-get remove x y z
To satisfy the dependency line, *all* packages in it need to be installed.
To "satisfy" a conflicts field (that is, there is a conflict), *any* of
the packages in it needs to be installed.
Carsten
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20120217191...@furrball.stateful.de
David Kalnischkies
Feb 17, 2012, 6:40:01 PM2/17/12
to
On Fri, Feb 17, 2012 at 19:53, Carsten Hey <car...@debian.org> wrote:
> * David Kalnischkies [2012-02-17 14:15 +0100]:
>> You generously left out the paragraph describing how APT should
>> detect that the package foo is in fact a library ...
>
> My impression was that you think very library centric. All I wrote was
> (in other words), that we should consider non-library packages as much
> as library packages, and I did not write nor implied that libraries
> should be handled in a different way.
>
>
>> Is 'apt-get remove foo+' then going to install all foo's or just one?
>
> "apt-get install g+++" is a weird syntax.
But its a syntax we support since basically ever and comes in handy
> * David Kalnischkies [2012-02-17 14:15 +0100]:
>> You generously left out the paragraph describing how APT should
>> detect that the package foo is in fact a library ...
>
> My impression was that you think very library centric. All I wrote was
> (in other words), that we should consider non-library packages as much
> as library packages, and I did not write nor implied that libraries
> should be handled in a different way.
>
>
>> Is 'apt-get remove foo+' then going to install all foo's or just one?
>
> "apt-get install g+++" is a weird syntax.
if you want to tell APT to not choose a specific alternative
(or disable specific recommends, …) without holds.
aptitude supports a few more of these modifiers (e.g. &) btw.
> The above shows that it seems to depend on the availability of
> foo:native if apt-get remove foo removes foo:foreign.
which 'changes' this behavior. As tendra is not available for amd64 its
a pretty fair guess that i386 was meant and - as the result would be an
error otherwise - install it.
It's removed with the same command s/install/remove/.
A similar guess isn't done for removes in case the native is not installed
(but available and foreigns are installed) as it is a destructive command
(beside that it would fail the s/remove/install/ test).
See also the arguments against the
"foo == foo:whatever provided that whatever is unique"
in Raphaels mail in thread [0] mentioned above.
And as i said, its not only about apt-get install/remove.
It would be nice to have an approach usable for the various
commands of apt-mark and apt-cache, too.
(bonus points if it doesn't break usage with dpkg completely)
> # dpkg -l | awk '$2=="clang"{print}'
> ii clang 3.0-5 Low-Level ...
>
> # dpkg -S bin/clang
> clang: /usr/bin/clang
> clang: /usr/bin/clang++
>
> # dpkg -r clang
> dpkg: warning: there's no installed package matching clang
If i understood the outcome of thread [1] above right dpkg
doesn't want to arch qualify packages for which only one package
can be meant. I personally think this is misfortune, but so it be.
APT can't do this as while you might have only one installed at a
time you can still jungle with different archs in one apt command
so we need differentiate here.
Long story short, 'apt-get remove clang' fails, yes, but you have
it installed with 'apt-get install clang:i386' so we are at least
consistent (see foo == foo:whatever).
I could have a look at printing a notice through to be nice…
> # dpkg -l | grep libzookeeper-st2
> ii libzookeeper-st2:amd64 3.3.4+dfsg1-3 Single ...
> ii libzookeeper-st2:i386 3.3.4+dfsg1-3 Single ...
>
> Unlike the above dpkg -l output showing the foreign clang package,
> libzookeeper-st2 is shown with the architecture appended.
It allows no arch as ~ 'installed' arch, but mostly only for backward-
compatibility as apt/squeeze obviously doesn't know about that new
requirement.
Best regards
David Kalnischkies
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 22, 2012, 5:30:03 AM2/22/12
to
to install that if it has a equal or greater version than the installed
linux-image-3.2.0-1-amd64:amd64.
Current apt behaviour is a bit strange there though:
mrvn@frosties:~% apt-cache policy acl
acl:
Installed: 2.2.51-4
Candidate: 2.2.51-5
Version table:
2.2.51-5 0
500 http://ftp.de.debian.org/debian/ sid/main amd64 Packages
*** 2.2.51-4 0
100 /var/lib/dpkg/status
mrvn@frosties:~% apt-cache policy acl:i386
acl:i386:
Installed: (none)
Candidate: 2.2.51-5
Version table:
2.2.51-5 0
500 http://ftp.de.debian.org/debian/ sid/main i386 Packages
I would expect something like:
mrvn@frosties:~% apt-cache policy acl
acl:
Installed: 2.2.51-4 amd64
Candidate: 2.2.51-5 amd64
Version table:
2.2.51-5 0
500 http://ftp.de.debian.org/debian/ sid/main amd64 Packages
499 http://ftp.de.debian.org/debian/ sid/main i386 Packages
*** 2.2.51-4 0
100 /var/lib/dpkg/status amd64
But it seems my patch to reduce the pin of non-native architectures is
not in current apt and policy doesn't list all archs for M-A:foreign
packages.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 23, 2012, 7:40:01 AM2/23/12
to
Russ Allbery <r...@debian.org> writes:
> Goswin von Brederlow <goswi...@web.de> writes:
>
>> Changing the name in the package would break tools that rely on the name
>> (like packages.debian.org extracting the Changelog). Also ugly.
>
> We control the tools; we can change the tools. Multiarch is a big deal.
> We weren't going to get through this without changing some tools. (One
> should not read that as my support of this specific alternative, as I've
> not decided there yet, but in general I think it's fair game to change our
> tools to support multiarch.)
One should not read that as my rejection of this specific alternative, as
I've not decided there yet, but in general I think it's fair game to
change our tools to support multiarch.
Problem I have is with the timeframe of such a change. While we can
change packages.d.o quite quickly given somebody willing to write the
patch we can not quickly change the tools in stable. And I really really
really do not want to have to wait yet another release cycle.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
> Goswin von Brederlow <goswi...@web.de> writes:
>
>> Changing the name in the package would break tools that rely on the name
>> (like packages.debian.org extracting the Changelog). Also ugly.
>
> We control the tools; we can change the tools. Multiarch is a big deal.
> We weren't going to get through this without changing some tools. (One
> should not read that as my support of this specific alternative, as I've
> not decided there yet, but in general I think it's fair game to change our
> tools to support multiarch.)
One should not read that as my rejection of this specific alternative, as
I've not decided there yet, but in general I think it's fair game to
change our tools to support multiarch.
Problem I have is with the timeframe of such a change. While we can
change packages.d.o quite quickly given somebody willing to write the
patch we can not quickly change the tools in stable. And I really really
really do not want to have to wait yet another release cycle.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87lintz...@frosties.localnet
Goswin von Brederlow
Feb 23, 2012, 8:40:02 AM2/23/12
to
Russ Allbery <r...@debian.org> writes:
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
>
> * dpkg re-adds the refcounting implementation for multiarch, but along
> with a Policy requirement that packages that are multiarch must only
> contain files in classes 1 and 2 above.
>
> * All packages that want to be multiarch: same have to move all generated
> documentation into a separate package unless the maintainer has very
> carefully checked that the generated documentation will be byte-for-byte
> identical even across minor updates of the documentation generation
> tools and when run at different times.
>
> * Lintian should recognize arch-qualified override files, and multiarch:
> same packages must arch-qualify their override files. debhelper
> assistance is desired for this.
I think that, provided the files are byte for byte identical across
architectures they need not be arch qualified. So they should be
refcounted and having non-identical files across archs should be
forbidden by policy. The maintainer must then resolve this by 1) making
the file identical across archs or 2) arch qualifying the name.
So lintian should support arch qualified names but policy should not
needlessly require them.
> * Policy prohibits arch-varying data files in multiarch: same packages
> except in arch-qualified paths.
>
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
>
> Please note that this is a bunch of work. I think the Lintian work is a
> good idea regardless, and it can start independently. I think the same is
> true of the binNMU changelog work, since this will address some
> long-standing issues with changelog handling in some situations, including
> resolving just how we're supposed to handle /usr/share/doc symlinks. But
> even with those aside, this is a lot of stuff that we need to agree on,
> and in some cases implement, in a fairly short timeline if this is going
> to make wheezy.
In case /usr/share/doc/pkg is a symlink the binNMU changelog should be
stored in the destination of the symlink. For this to work the (binNMU)
changelog should be both arch and package qualified
(changelog.Debain.bin-pkg.arch). This would allow libfoo:any,
foo-bin:any + foo-common:all to be binNMUed. Without the package
qualifier the libfoo and foo-bin package would both contain
/usr/share/doc/foo-common/changelog.Debian.arch and produce a file
overwrite error.
After a binNMU (on amd64) the following files would exist:
foo-bin: /usr/share/doc/foo-bin -> foo-common
foo-bin: /usr/share/doc/foo-common/changelog.Debian.foo-bin.amd64
foo-common: /usr/share/doc/foo-common/changelog.Debian
libfoo: /usr/share/doc/foo-common/changelog.Debian.libfoo.amd64
libfoo: /usr/share/doc/libfoo -> foo-common
The binNMU changelogs would be identical wasting a little disk and
mirror space and bandwidth but that is probably unavoidable.
One way to reduce the overhead would be to split the binNMU entry into a
separate changelog:
foo-bin: /usr/share/doc/foo-bin -> foo-common
foo-bin: /usr/share/doc/foo-common/changelog.binNMU.foo-bin.amd64
foo-common: /usr/share/doc/foo-common/changelog.Debian
libfoo: /usr/share/doc/foo-common/changelog.binNMU.libfoo.amd64
libfoo: /usr/share/doc/libfoo -> foo-common
The binNMU changelogs would be just one entry each, the reason for the
binNMU.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
> If this is comprehensive, then I propose the following path forward, which
> is a mix of the various solutions that have been discussed:
>
> * dpkg re-adds the refcounting implementation for multiarch, but along
> with a Policy requirement that packages that are multiarch must only
> contain files in classes 1 and 2 above.
>
> * All packages that want to be multiarch: same have to move all generated
> documentation into a separate package unless the maintainer has very
> carefully checked that the generated documentation will be byte-for-byte
> identical even across minor updates of the documentation generation
> tools and when run at different times.
>
> * Lintian should recognize arch-qualified override files, and multiarch:
> same packages must arch-qualify their override files. debhelper
> assistance is desired for this.
I think that, provided the files are byte for byte identical across
architectures they need not be arch qualified. So they should be
refcounted and having non-identical files across archs should be
forbidden by policy. The maintainer must then resolve this by 1) making
the file identical across archs or 2) arch qualifying the name.
So lintian should support arch qualified names but policy should not
needlessly require them.
> * Policy prohibits arch-varying data files in multiarch: same packages
> except in arch-qualified paths.
>
> * The binNMU process is changed to add the binNMU changelog entry to an
> arch-qualified file (changelog.Debian.arch, probably). We need to
> figure out what this means if the package being binNMU'd has a
> /usr/share/doc/<package> symlink to another package, though; it's not
> obvious what to do here.
>
> Please note that this is a bunch of work. I think the Lintian work is a
> good idea regardless, and it can start independently. I think the same is
> true of the binNMU changelog work, since this will address some
> long-standing issues with changelog handling in some situations, including
> resolving just how we're supposed to handle /usr/share/doc symlinks. But
> even with those aside, this is a lot of stuff that we need to agree on,
> and in some cases implement, in a fairly short timeline if this is going
> to make wheezy.
In case /usr/share/doc/pkg is a symlink the binNMU changelog should be
stored in the destination of the symlink. For this to work the (binNMU)
changelog should be both arch and package qualified
(changelog.Debain.bin-pkg.arch). This would allow libfoo:any,
foo-bin:any + foo-common:all to be binNMUed. Without the package
qualifier the libfoo and foo-bin package would both contain
/usr/share/doc/foo-common/changelog.Debian.arch and produce a file
overwrite error.
After a binNMU (on amd64) the following files would exist:
foo-bin: /usr/share/doc/foo-bin -> foo-common
foo-bin: /usr/share/doc/foo-common/changelog.Debian.foo-bin.amd64
foo-common: /usr/share/doc/foo-common/changelog.Debian
libfoo: /usr/share/doc/foo-common/changelog.Debian.libfoo.amd64
libfoo: /usr/share/doc/libfoo -> foo-common
The binNMU changelogs would be identical wasting a little disk and
mirror space and bandwidth but that is probably unavoidable.
One way to reduce the overhead would be to split the binNMU entry into a
separate changelog:
foo-bin: /usr/share/doc/foo-bin -> foo-common
foo-bin: /usr/share/doc/foo-common/changelog.binNMU.foo-bin.amd64
foo-common: /usr/share/doc/foo-common/changelog.Debian
libfoo: /usr/share/doc/foo-common/changelog.binNMU.libfoo.amd64
libfoo: /usr/share/doc/libfoo -> foo-common
The binNMU changelogs would be just one entry each, the reason for the
binNMU.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87d395z...@frosties.localnet
Goswin von Brederlow
Feb 23, 2012, 9:00:02 AM2/23/12
to
Russ Allbery <r...@debian.org> writes:
> Carsten Hey <car...@debian.org> writes:
>> * Russ Allbery [2012-02-16 14:55 -0800]:
>
>>> Every file that differs has to be fixed in the current multi-arch plan.
>>> Documentation that contains its build date is going to need to be split
>>> out into a separate -docs package.
>
>> I doubt that ftpmaster would be happy about -doc packages that contain
>> just a few small man pages.
>
> I think they'll be okay with it when it's the cost of reasonable
> multiarch.
>
> I feel fairly strongly that it isn't sane to have a file overlap when the
> file doesn't match. You then lose the error detection when there are real
> problems, and I don't trust any of us, myself included, to correctly tag
> files where it doesn't matter.
>
> On this front, I agree with Guillem: some amount of package splitting is
> fine, and package splitting instead of additional complexity, like tagging
> files that are allowed to vary, looks like a good tradeoff to me. The
> splitting that I'm worried about is where the files are tightly coupled,
> which is not the case for development man pages that are now in -dev
> packages.
+1.
> Carsten Hey <car...@debian.org> writes:
>> * Russ Allbery [2012-02-16 14:55 -0800]:
>
>>> Every file that differs has to be fixed in the current multi-arch plan.
>>> Documentation that contains its build date is going to need to be split
>>> out into a separate -docs package.
>
>> I doubt that ftpmaster would be happy about -doc packages that contain
>> just a few small man pages.
>
> I think they'll be okay with it when it's the cost of reasonable
> multiarch.
>
> I feel fairly strongly that it isn't sane to have a file overlap when the
> file doesn't match. You then lose the error detection when there are real
> problems, and I don't trust any of us, myself included, to correctly tag
> files where it doesn't matter.
>
> On this front, I agree with Guillem: some amount of package splitting is
> fine, and package splitting instead of additional complexity, like tagging
> files that are allowed to vary, looks like a good tradeoff to me. The
> splitting that I'm worried about is where the files are tightly coupled,
> which is not the case for development man pages that are now in -dev
> packages.
I find the argument about past experience with splitting packages and
upstream later changing files so they become arch dependent convincing
in this regard. Refcounting and no exception for file differences seems
to be the best way.
>> debianutils uses a special make target 'prebuild' in debian/rules to
>> update build system related files and PO files before the actual source
>> package is built.
>
>> This basic idea also could be used to build problematic documentation
>> files on the maintainers computer before he/she builds the package. The
>> other targets would then install the prebuilt documentation into the
>> package without the need to build it first. A proper dependency on
>> debian/$prebuilt_doc could ensure that maintainers do not forget to run
>> debian/rules prebuild.
>
>> If maintainers choose to use such a target, suggesting a common name for
>> it in the developers reference could be reasonable.
>
> That's an interesting idea. That's very similar to what I already do as
> upstream (I build POD-generated man pages from my autogen script, and in
> Debian packaging don't bother to regenerate them).
You are probably not the only one doing something like this. Who else
does this? What automatism do you have in your debian/rules to help?
Lets see if we can get a good set of examples to work out some
recommendations.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 23, 2012, 9:10:02 AM2/23/12
to
Josselin Mouette <jo...@debian.org> writes:
> Le lundi 13 février 2012 à 22:43 -0800, Russ Allbery a écrit :
>> There's been a lot of discussion of this, but it seems to have been fairly
>> inconclusive. We need to decide what we're doing, if anything, for wheezy
>> fairly soon, so I think we need to try to drive this discussion to some
>> concrete conclusions.
>
> Thank you very much for your constructive work.
>
>> 3. Generated documentation. Here's where I think refcounting starts
>> failing.
>
> So we need to move a lot of documentation generated with gtk-doc or
> doxygen from -dev packages to -doc packages. But it really seems an
> acceptable tradeoff between the amount of work required and the
> cleanness of the solution.
>
>> Does this seem comprehensive to everyone? Am I missing any cases?
>
> Are there any cases of configuration files in /etc that vary across
> architectures? Think of stuff like ld.so.conf, where some plugins or
> library path is coded in a configuration file.
Generally conffiles in library packages is a violation of policy
8.2. They would create a file overwrite conflict if the SONAME
changes. Putting the conffile into the -common package is a good
solution.
There are some exceptions where the conffiles are version qualified.
E.g.: libgtk2.0-0: /etc/gtk-2.0/im-multipress.conf
For conffiles that vary across architectures the path or name must
include the multiarch triplet.
E.g: libc6: /etc/ld.so.conf.d/x86_64-linux-gnu.conf
(Note: this is actualy a violation of policy 8.2 and needs to be fixed
when we get a libc7).
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/874nuhz...@frosties.localnet
> Le lundi 13 février 2012 à 22:43 -0800, Russ Allbery a écrit :
>> There's been a lot of discussion of this, but it seems to have been fairly
>> inconclusive. We need to decide what we're doing, if anything, for wheezy
>> fairly soon, so I think we need to try to drive this discussion to some
>> concrete conclusions.
>
> Thank you very much for your constructive work.
>
>> 3. Generated documentation. Here's where I think refcounting starts
>> failing.
>
> So we need to move a lot of documentation generated with gtk-doc or
> doxygen from -dev packages to -doc packages. But it really seems an
> acceptable tradeoff between the amount of work required and the
> cleanness of the solution.
>
>> Does this seem comprehensive to everyone? Am I missing any cases?
>
> Are there any cases of configuration files in /etc that vary across
> architectures? Think of stuff like ld.so.conf, where some plugins or
> library path is coded in a configuration file.
Generally conffiles in library packages is a violation of policy
8.2. They would create a file overwrite conflict if the SONAME
changes. Putting the conffile into the -common package is a good
solution.
There are some exceptions where the conffiles are version qualified.
E.g.: libgtk2.0-0: /etc/gtk-2.0/im-multipress.conf
For conffiles that vary across architectures the path or name must
include the multiarch triplet.
E.g: libc6: /etc/ld.so.conf.d/x86_64-linux-gnu.conf
(Note: this is actualy a violation of policy 8.2 and needs to be fixed
when we get a libc7).
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 23, 2012, 9:20:02 AM2/23/12
to
Joey Hess <jo...@debian.org> writes:
> Goswin von Brederlow wrote:
>> pkg:arch will still be unique and the dpkg/apt output will use the
>> architecture where required for uniqueness. So I think that after some
>> getting used to it it will be clear enough again.
>
> Goswin von Brederlow wrote:
>> pkg:arch will still be unique and the dpkg/apt output will use the
>> architecture where required for uniqueness. So I think that after some
>> getting used to it it will be clear enough again.
>
> Here are a few examples of the problems I worry about. I have not
> verified any of them, and they're clearly biased toward code I am
> familiar with, which suggests there are many other similar problems.
>
> * Puppet not only installs packages, it may remove them. A puppet config
> that does dpkg --purge foo will fail if multiarch is enabled, now
> verified any of them, and they're clearly biased toward code I am
> familiar with, which suggests there are many other similar problems.
>
> * Puppet not only installs packages, it may remove them. A puppet config
> that does dpkg --purge foo will fail if multiarch is enabled, now
> it needs to find and remove foo:*
>
>
> * dpkg-repack pkg:arch will create a package with that literal name (or fail)
>
>
> * dpkg-reconfigure probably can't be used with M-A same packages.
> debconf probably generally needs porting to multiarch.
>
> * tasksel uses dpkg --query to work out if a task's dependencies are
> debconf probably generally needs porting to multiarch.
>
> installed. In the event that a library is directly part of a task,
> this will fail when multiarch is enabled.
>
> * Every piece of documentation that gives commands lines manipulating
> library packages is potentially broken.
>
> Seems like we need a release where multiarch is classed as an
> experimental feature, which when enabled can break the system. But the
> Seems like we need a release where multiarch is classed as an
> sort of problems above are the easy to anticipate ones; my real worry is
> the unanticipated classes of problems. Especially if we find intractable
> problems or levels of complexity introduced by dropping the unique
> package name invariant.
>
> the unanticipated classes of problems. Especially if we find intractable
> problems or levels of complexity introduced by dropping the unique
> package name invariant.
>
> My nightmare scenario is that we release with multiarch, discover that
> it's a net negative for our users (proprietary software on amd64 aside,
> nearly all the benefits are to developers AFAICS), and are stuck with it.
The specs were initialy written in such a way that single arch systems
> it's a net negative for our users (proprietary software on amd64 aside,
> nearly all the benefits are to developers AFAICS), and are stuck with it.
would not change, that multiarch packages would keep functioning with a
mono-arch apt/dpkg and I think this was preserved so far.
If all interface changes foolow that idea then worst case tools will not
work in a multiarch configuration but still work in a monoarch
configuration. So let multiarch be experimental and only for developers
and risk takers. That is already a huge number of people.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 23, 2012, 9:30:04 AM2/23/12
to
Russ Allbery <r...@debian.org> writes:
> I think it would be better to have a world in which all the architectures
> of the foo package on the system have to have the same version, because
> then you don't have to treat foo:i386 and foo:amd64 like they're separate
> packages. The list of installed architectures is an *attribute* of the
> package. A package is still either installed or not installed, but when
> it's installed, it can be installed for one or more architectures. But if
> it's installed for multiple architectures, those architectures are always
> upgraded together and always remain consistent. That avoids all weirdness
> of having a package work differently because the version varies depending
> on the ABI, and it significantly simplifies the mental model behind the
> package.
In such a world architecture all could also be considered another
> I think it would be better to have a world in which all the architectures
> of the foo package on the system have to have the same version, because
> then you don't have to treat foo:i386 and foo:amd64 like they're separate
> packages. The list of installed architectures is an *attribute* of the
> package. A package is still either installed or not installed, but when
> it's installed, it can be installed for one or more architectures. But if
> it's installed for multiple architectures, those architectures are always
> upgraded together and always remain consistent. That avoids all weirdness
> of having a package work differently because the version varies depending
> on the ABI, and it significantly simplifies the mental model behind the
> package.
architecture. And then foo:i386, foo:amd64 and foo:all could be
coinstallable. That would mean that files shared between architectures
could be moved into foo:all and foo:any could implicitly depend on
foo:all. The benefit of this over foo-common would be that apt-cache
search, apt-cache policy, aptitude, dpkg --remove, ... would only have
one package (foo) instead of 2 (foo + foo-common).
This has been previously suggested too but has been droped because it
would be incompatible with existing systems (i.e. monoarch dpkg couldn't
install packages from such a world).
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Feb 23, 2012, 9:40:02 AM2/23/12
to
Package: gcc
Source: gcc-defaults (1.93ubuntu1)
Version: 4:4.4.3-1ubuntu1
What would the version be for a binNMU of gcc-defaults? I think it would
be
Package: gcc
Source: gcc-defaults (1.93ubuntu1)
Version: 4:4.4.3-1ubuntu1+b1
What we want is for apt/dpkg to consider this to be compatible with
"4:4.4.3-1ubuntu1".
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Raphael Hertzog
Feb 23, 2012, 10:00:03 AM2/23/12
to
On Thu, 23 Feb 2012, Goswin von Brederlow wrote:
> > Package: 3depict
> > Source: 3depict (0.0.9-1)
> > Version: 0.0.9-1+b1
>
> > Package: 3depict
> > Source: 3depict (0.0.9-1)
> > Version: 0.0.9-1+b1
>
> Except that doesn't have to work (sorry for the ubuntu part):
>
> Package: gcc
> Source: gcc-defaults (1.93ubuntu1)
> Version: 4:4.4.3-1ubuntu1
>
> What would the version be for a binNMU of gcc-defaults?
What about trying it?
>
> Package: gcc
> Source: gcc-defaults (1.93ubuntu1)
> Version: 4:4.4.3-1ubuntu1
>
> What would the version be for a binNMU of gcc-defaults?
$ head -n 1 debian/changelog
gcc-defaults (1.112+b1) unstable; urgency=low
$ debuild -us -uc
[...]
$ dpkg -I ../gcc_4.6.2-4+b1_i386.deb
[...]
Package: gcc
Source: gcc-defaults (1.112)
Version: 4:4.6.2-4+b1
In any case, the fact that the source version is unrelated to the binary
version doesn't change anything to the requirement. We just want to ensure
that all (M-A: same) co-installable packages are synchronized at the same
version (either the source version, or the binary version but stripped
from its bin-NMU suffix).
Cheers,
--
Raphaël Hertzog ◈ Debian Developer
Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/
--
To UNSUBSCRIBE, email to debian-dp...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012022314...@rivendell.home.ouaza.com
Guillem Jover
Feb 29, 2012, 3:00:03 PM2/29/12
to
On Wed, 2012-02-15 at 16:41:21 +0000, Ian Jackson wrote:
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > [...] But trying to workaround this by coming
> > up with stacks of hacked up solutions [...]
>
> I disagree with your tendentious phrasing. The refcnt feature is not
> a "hacked up solution" (nor a stack of them). It is entirely normal
> in Debian core tools (as in any substantial piece of software serving
> a lot of diverse needs) to have extra code to make it easier to deploy
> or use in common cases simpler.
All along this thread, when referring to the additional complexity and
the additional hacks, I've not been talking about the refcnt'ing at
all, but to all the other "fixes" needed to make it a workable solution.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > [...] But trying to workaround this by coming
> > up with stacks of hacked up solutions [...]
>
> I disagree with your tendentious phrasing. The refcnt feature is not
> a "hacked up solution" (nor a stack of them). It is entirely normal
> in Debian core tools (as in any substantial piece of software serving
> a lot of diverse needs) to have extra code to make it easier to deploy
> or use in common cases simpler.
All along this thread, when referring to the additional complexity and
the additional hacks, I've not been talking about the refcnt'ing at
all, but to all the other "fixes" needed to make it a workable solution.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012022919...@gaara.hadrons.org
Guillem Jover
Feb 29, 2012, 9:10:01 PM2/29/12
to
On Thu, 2012-02-16 at 10:43:53 -0800, Russ Allbery wrote:
> I was thinking more about this, and I was finally able to put a finger on
> why I don't like package splitting as a solution.
>
> We know from prior experience with splitting packages for large
> arch-independent data that one of the more common mistakes that we'll make
> is to move the wrong files: to put into the arch-independent package a
> file that's actually arch-dependent.
This was brought up by Steve in the thread, my reply:
> I was thinking more about this, and I was finally able to put a finger on
> why I don't like package splitting as a solution.
>
> We know from prior experience with splitting packages for large
> arch-independent data that one of the more common mistakes that we'll make
> is to move the wrong files: to put into the arch-independent package a
> file that's actually arch-dependent.
<http://lists.debian.org/debian-devel/2012/02/msg00497.html>
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 29, 2012, 9:20:01 PM2/29/12
to
Guillem Jover <gui...@debian.org> writes:
> On Thu, 2012-02-16 at 10:43:53 -0800, Russ Allbery wrote:
>> I was thinking more about this, and I was finally able to put a finger
>> on why I don't like package splitting as a solution.
>> We know from prior experience with splitting packages for large
>> arch-independent data that one of the more common mistakes that we'll
>> make is to move the wrong files: to put into the arch-independent
>> package a file that's actually arch-dependent.
> This was brought up by Steve in the thread, my reply:
> <http://lists.debian.org/debian-devel/2012/02/msg00497.html>
Thanks for the pointer, Guillem, but I'm afraid I don't think this reply
> On Thu, 2012-02-16 at 10:43:53 -0800, Russ Allbery wrote:
>> I was thinking more about this, and I was finally able to put a finger
>> on why I don't like package splitting as a solution.
>> We know from prior experience with splitting packages for large
>> arch-independent data that one of the more common mistakes that we'll
>> make is to move the wrong files: to put into the arch-independent
>> package a file that's actually arch-dependent.
> This was brought up by Steve in the thread, my reply:
> <http://lists.debian.org/debian-devel/2012/02/msg00497.html>
addresses my concerns. See the specific enumeration of things that we
would have to split, and the ways in which they can break. I think the
issue with C headers is particularly severe.
I don't think this mirrors an existing problem. The sorts of things we
split into arch: all packages are nowhere near as intrusive or as tightly
coupled as the things we're talking about splitting to avoid refcounting;
for example, right now, splitting out C headers into arch: all packages is
very rare. The sort of package splitting that we would do to avoid
refcounting would run a serious risk of introducing substantial new
problems that we don't currently have.
The situation with refcounting seems much less fragile than the situation
without refcounting to me. Refcounting puts the chance of error in the
right place (on people who want to use the new feature, since the
situation will not change for users who continue using packages the way
they do today), provides a clear error message rather than silent
corruption, and fails safely (on any inconsistency) rather than appearing
to succeed in situations that are not consistent. Those are all good
design principles to have.
I think the principle of not changing things for people who are not using
multiarch is particularly important, and is inconsistent with either
package splitting or with moving files into arch-qualified paths. We
should attempt to adopt new features in a way that puts most of the risk
on the people who are making use of the new features, and tries to be as
safe as possible for existing users. I agree that we should not pursue
that to an extreme that leads to an unmaintainable system, but I don't
believe refcounting has that problem.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/874nu9v...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Guillem Jover
Feb 29, 2012, 10:10:03 PM2/29/12
to
On Wed, 2012-02-15 at 16:32:38 -0800, Russ Allbery wrote:
> Guillem Jover <gui...@debian.org> writes:
> > If packages have to be split anyway to cope with the other cases, then
> > the number of new packages which might not be needed otherwise will be
> > even smaller than the predicted amount, at which point it makes even
> > less sense to support refcnt'ing.
>
> I don't think the package count is really the interesting metric here,
> unless the number of introduced packages is very large (see below about
> -dev packages). I'm more concerned with maintainer time and with
> dependency complexity, and with the known problems that we introduce
> whenever we take tightly-coupled files and separate them into independent
> packages.
Well, people have been using the amount of packages as a metric, I've
just been trying to counter it. It also in a way represents the amount
of work needed.
About tightly-coupled files, they can cause serious issues also with
refcounting, consider that there's always going to be a point when
unpacking one of the new instances will have a completely different
vesion than the other already unpacked instance(s). So packages could
stop working for a long time if say unpacked libfoo0:i386 1.0 has
file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the
file didn't change name (arguably this could be considered an upstream
problem, depending on the situation), this would be particularly
problematic for pseudo-essential packages.
> I just posted separately about version lockstep: I think this is a
> feature, not a bug, in our multiarch implementation. I think this is the
> direction we *should* go, because it reduces the overall complexity of the
> system. [...]
I've replied to that separately, in any case I think the best compromise
would be to add version lockstep to dpkg, but not refcounting. Because
the first is a restriction that can always be lifted if it's confirmed
to cause issues (which I think it will), and the second can always be
added later because it's something that allows things not permitted
previously.
But at this point it seems I'm alone in thinking that refcounting has
more negative implications than positive ones, and I cannot get myself
to care enough any longer to push for this. So some weeks ago I added
back both those things to my local repo.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2012030103...@gaara.hadrons.org
> Guillem Jover <gui...@debian.org> writes:
> > If packages have to be split anyway to cope with the other cases, then
> > the number of new packages which might not be needed otherwise will be
> > even smaller than the predicted amount, at which point it makes even
> > less sense to support refcnt'ing.
>
> I don't think the package count is really the interesting metric here,
> unless the number of introduced packages is very large (see below about
> -dev packages). I'm more concerned with maintainer time and with
> dependency complexity, and with the known problems that we introduce
> whenever we take tightly-coupled files and separate them into independent
> packages.
Well, people have been using the amount of packages as a metric, I've
just been trying to counter it. It also in a way represents the amount
of work needed.
About tightly-coupled files, they can cause serious issues also with
refcounting, consider that there's always going to be a point when
unpacking one of the new instances will have a completely different
vesion than the other already unpacked instance(s). So packages could
stop working for a long time if say unpacked libfoo0:i386 1.0 has
file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the
file didn't change name (arguably this could be considered an upstream
problem, depending on the situation), this would be particularly
problematic for pseudo-essential packages.
> I just posted separately about version lockstep: I think this is a
> feature, not a bug, in our multiarch implementation. I think this is the
> direction we *should* go, because it reduces the overall complexity of the
> system. [...]
I've replied to that separately, in any case I think the best compromise
would be to add version lockstep to dpkg, but not refcounting. Because
the first is a restriction that can always be lifted if it's confirmed
to cause issues (which I think it will), and the second can always be
added later because it's something that allows things not permitted
previously.
But at this point it seems I'm alone in thinking that refcounting has
more negative implications than positive ones, and I cannot get myself
to care enough any longer to push for this. So some weeks ago I added
back both those things to my local repo.
regards,
guillem
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Russ Allbery
Feb 29, 2012, 10:20:01 PM2/29/12
to
Guillem Jover <gui...@debian.org> writes:
> About tightly-coupled files, they can cause serious issues also with
> refcounting, consider that there's always going to be a point when
> unpacking one of the new instances will have a completely different
> vesion than the other already unpacked instance(s). So packages could
> stop working for a long time if say unpacked libfoo0:i386 1.0 has
> file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the file
> didn't change name (arguably this could be considered an upstream
> problem, depending on the situation), this would be particularly
> problematic for pseudo-essential packages.
Yes, I agree. Refcounting does complicate the upgrade situation, since
> About tightly-coupled files, they can cause serious issues also with
> refcounting, consider that there's always going to be a point when
> unpacking one of the new instances will have a completely different
> vesion than the other already unpacked instance(s). So packages could
> stop working for a long time if say unpacked libfoo0:i386 1.0 has
> file-format-0, but new libfoo0:amd64 4.0 has file-format-2, and the file
> didn't change name (arguably this could be considered an upstream
> problem, depending on the situation), this would be particularly
> problematic for pseudo-essential packages.
you really want to upgrade all installed architectures in lockstep to
ensure that we maintain as many of the guarantees of file consistency as
we do now with single-arch upgrades.
> I've replied to that separately, in any case I think the best compromise
> would be to add version lockstep to dpkg, but not refcounting. Because
> the first is a restriction that can always be lifted if it's confirmed
> to cause issues (which I think it will), and the second can always be
> added later because it's something that allows things not permitted
> previously.
I said that introducing refcounting doesn't make me nervous. You're
right, it's something that's going to be very difficult to back out of if
we decide it's a mistake.
I do think it's the best solution to a complex set of issues, but we're
going to have to use it in conjunction with pretty tight version lockstep
to avoid problems with file inconsistency.
> But at this point it seems I'm alone in thinking that refcounting has
> more negative implications than positive ones, and I cannot get myself
> to care enough any longer to push for this. So some weeks ago I added
> back both those things to my local repo.
rather have us all agree. But I do want to wholeheartedly second
Christian's thanks for all your work on dpkg in the middle of a really
difficult situation, and your willingness to make compromises like this
even when you think they're the wrong technical decision. That's really
hard to do, and I think it's also very admirable.
If this all turns out to be a horrible mistake, I for one will try to help
us back out of it as needed, to put my resources where my advocacy has
been.
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87sjhtu...@windlord.stanford.edu
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Marco d'Itri
Mar 1, 2012, 5:00:01 AM3/1/12
to
On Mar 01, Russ Allbery <r...@debian.org> wrote:
> The situation with refcounting seems much less fragile than the situation
> without refcounting to me.
I totally agree.
> The situation with refcounting seems much less fragile than the situation
> without refcounting to me.
Also, why does refcounting have to be "perfect"?
What would break if it did not actually check that the two files
provided by the same package for different architectures are identical?
--
ciao,
Marco
Russ Allbery
Mar 1, 2012, 1:40:02 PM3/1/12
to
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Mar 4, 2012, 1:30:02 AM3/4/12
to
m...@Linux.IT (Marco d'Itri) writes:
the stability advantage.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Goswin von Brederlow
Mar 4, 2012, 1:40:01 AM3/4/12
to
be marginally for refcounting:
The same situation would occur with libfoo0:i386 1.0, libfoo0:amd64 4.0
and libfoo0-common:all 2.0. But now even worse because you have 3
versions that can be out-of-sync.
Actualy if the file is shipped in the package then ref counting would
automatically detect the difference in contents and fail to install the
new libfoo0:amd64 4.0. And if the file is not shipped in the package
then ref counting has no effect on it. Again ref counting comes out
better.
Ref counting would catch some of those cases but not all and it never
makes it worse. What solves this problem is the same version requirement
or simply adding Breaks: libfoo0 (<< 4.0~) to libfoo0:* 4.0. The only
point you've made is that ref counting isn't a magic bullet that brings
us world peace.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Marco d'Itri
Mar 4, 2012, 7:30:01 AM3/4/12
to
On Mar 04, Goswin von Brederlow <goswi...@web.de> wrote:
> > Also, why does refcounting have to be "perfect"?
> > What would break if it did not actually check that the two files
> > provided by the same package for different architectures are identical?
> Everything that can go wrong when splitting packages. You would loose
> the stability advantage.
Yes, but with much less work needed by maintainers. So it still looks
> > Also, why does refcounting have to be "perfect"?
> > What would break if it did not actually check that the two files
> > provided by the same package for different architectures are identical?
> Everything that can go wrong when splitting packages. You would loose
> the stability advantage.
like a better option to me.
--
ciao,
Marco
0 new messages