Status of new source formats project
Raphael Hertzog
I updated the wiki page listing the status of this project:
http://wiki.debian.org/Projects/DebSrc3.0
During debconf I used part of my time to push this project forward.
1/ Lucas did again a rebuild of the archive to discover problems that will
appear if all packages are converted to 3.0 (quilt) format. I processed
200 failed build logs and filed some new wishlist bugs to prepare for a
global transition (and fixed a sid-only bug in dpkg-source detected thanks
to this rebuild).
http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=her...@debian.org;tag=3.0-quilt-by-default
2/ I prepared some test source packages covering most interesting cases
to try out with various tools, I checked the most important ones (apt-get
source, sbuild) and they do cope with it. I encourage you to test all
other tools and report your success/failures here (and please file bug for
the failures).
http://people.debian.org/~hertzog/packages/debsrc3.0/
3/ I discussed with Jï¿œrg Jaspert and Mark Hymers the status of my patch
against dak. Currently they plan to merge my patch after they have
finished to update dak to use sqlalchemy, but given that this is an
intrusive change they will have to test it carefully and Jï¿œrg expected
that it will still be 2 to 3 months until they can integrate my patch. :-(
Both patches/branches are going to conflict so someone will have to handle
the merge conflict whatever the merge order is. I offered them to handle
it myself even if my patch is merged now (without conflicting with
anything since the sqlalchemy one is not yet ready/merged). The sqlalchemy
rewrite is not yet public but Mark hopes to publish it this week-end (1-2
Aug). After my upcoming vacation, and if I still can't convince ftpmasters
to merge my working branch immediately I'll create a new branch of my
patch merged with the new sqlalchemy branch so that they have a chance
to push both changes at the same time (or in the order that they want).
This dak modification is the main blocker point for this project to go
forward (and has been so for quite some time already).
4/ I discussed with Luk Claes if anything had to be done on the buildd
side. There's not much to be done there since the new sbuild is working
well with new source formats but there are still some buildd using the old
version. Those have to be updated to the new sbuild and this process has
been ongoing for quite some time already. I pinged some of the concerned
buildd maintainers and provided sample buildd logs generated by the new
sbuild with the new source formats so that they can adapt their build logs
processing scripts immediately.
5/ In all cases, I hope we will enable support of new source formats in
squeeze so that individual packages can switch to it (in particular bigger
packages that would benefit from the multi-tarball thingie). But depending
on the time left before the freeze, we might not start a global transition
to the new formats (i.e. I would not modify dpkg-source to produce newer
source formats when there's no debian/source/format).
Cheers,
--
Raphaï¿œl Hertzog
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Charles Plessy
>
> I updated the wiki page listing the status of this project:
> http://wiki.debian.org/Projects/DebSrc3.0
Bonjour Raphaël,
first of all, thank you for making things progress for the support of a
next-generation source format. In the case of programs distributed as a bzipped
tar archive, it will save us some time, allow to delete a few README.source and
get-orig-source files, and simplifiy our rules files.
But actually, among the programs that are not distributed upstream in a tar.gz
format, we in the Debian Med team have as many zip cases as bzip2. Do you think
that it would be possible to support zip format (i.e. .zip, .jar and .xpi
extensions) for Debian source packages version 3?
I understand that it would require to change a few things in the Dpkg perl
modules, as they assume that the original archives are always using tar. I have
quite poor programming skills, but if you and others do not have time but
nevertheless are interested by such a modification, I can try to work on a
patch (possibly using libarchive-any-perl). In that case, can you tell me what
kind of timeline would fit?
Another question that I would like to ask is on the auto-patching
functionality. One of the programs we package, EMBOSS, is released once a year
every 15th of July, and other updates are made via patches. Currently it is
possible to just give the patch to quilt to apply it. With the new source
format ‘3.0 (quilt)’, it is not possible. Do you think it would be possible to
change this behavior or at least to disable the auto-patching facilities? It
would be of course easy to convert the patch, but I would really prefer to stay
as identical to upstream's materials as possible.
Have a nice sunday,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Raphael Hertzog
> But actually, among the programs that are not distributed upstream in a tar.gz
> format, we in the Debian Med team have as many zip cases as bzip2. Do you think
> that it would be possible to support zip format (i.e. .zip, .jar and .xpi
> extensions) for Debian source packages version 3?
Not really. First of all, that support should have been implemented with
the rest so that it's available in lenny already. Then many options and
features rely on the fact that tar is used. So it would probably requires
addition of a supplementary abstraction/indirection.
> I understand that it would require to change a few things in the Dpkg perl
> modules, as they assume that the original archives are always using tar. I have
> quite poor programming skills, but if you and others do not have time but
> nevertheless are interested by such a modification, I can try to work on a
> patch (possibly using libarchive-any-perl). In that case, can you tell me what
> kind of timeline would fit?
I try to avoid depending on modules outside of perl-modules. I might
consider a patch if it's clean enough but then it will be not easy to
integrate it cleanly.
Then it will also require supplementary modifications to dak to allow .zip
and I don't know if ftpmasters would like it, you should better check this
before-hand.
> Another question that I would like to ask is on the auto-patching
> functionality. One of the programs we package, EMBOSS, is released once a year
> every 15th of July, and other updates are made via patches. Currently it is
> possible to just give the patch to quilt to apply it. With the new source
> format ‘3.0 (quilt)’, it is not possible. Do you think it would be possible to
> change this behavior or at least to disable the auto-patching facilities?
Why would it be impossible? You can modify the patch series to integrate
whatever set of patches that you want...
You can avoid using auto-patching of course, just put your patches
somewhere else than debian/patches/.
> It would be of course easy to convert the patch, but I would really
> prefer to stay as identical to upstream's materials as possible.
How are patches distributed and what kind of conversions is needed?
Cheers,
--
Raphaël Hertzog
Charles Plessy
> On Sun, 02 Aug 2009, Charles Plessy wrote:
About zip support:
> > But actually, among the programs that are not distributed upstream in a tar.gz
> > format, we in the Debian Med team have as many zip cases as bzip2. Do you think
> > that it would be possible to support zip format (i.e. .zip, .jar and .xpi
> > extensions) for Debian source packages version 3?
>
> Not really. First of all, that support should have been implemented with
> the rest so that it's available in lenny already. Then many options and
> features rely on the fact that tar is used. So it would probably requires
> addition of a supplementary abstraction/indirection.
I see that .bzip2 and .lzma are also supported compression methods for the 3.0
(native) format as well as for the binary packages. But I do not think it would
be useful to add zip to this list. It seems to me that the only thing needed is
the capacity to unpack the original upstream sources. In that case there would
not be a need for a Lenny support, isn't it?
About patches not accepted by dpkg-dev:
> > It would be of course easy to convert the patch, but I would really
> > prefer to stay as identical to upstream's materials as possible.
>
> How are patches distributed and what kind of conversions is needed?
The patch is available from the upstream FTP server and it just works to drop
it in debian/patches, the only modification being of course to add it a nice
DEP-3 header.
The patch is not in unified format, which causes the failure of
dpkg-buildpackage. It is trivial to refresh it with quilt to the unified
format, but this introduces a divergence with upstream that I would prefer to
avoid, because it makes it difficult for others to verify that we did not
modify it. Can you support in the format ’3.0 (quilt)’ patches that are
accepted by default quilt installation?
Cheers,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Emilio Pozuelo Monfort
> I see that .bzip2 and .lzma are also supported compression methods for the 3.0
> (native) format as well as for the binary packages. But I do not think it would
> be useful to add zip to this list. It seems to me that the only thing needed is
> the capacity to unpack the original upstream sources. In that case there would
> not be a need for a Lenny support, isn't it?
You need it to be supported in stable before using it in unstable. So at best,
you would need to implement it in Squeeze and wait for Squeeze+1 to use it.
Cheers,
Emilio
Russ Allbery
> On Sun, 02 Aug 2009, Charles Plessy wrote:
>> But actually, among the programs that are not distributed upstream in a
>> tar.gz format, we in the Debian Med team have as many zip cases as
>> bzip2. Do you think that it would be possible to support zip format
>> (i.e. .zip, .jar and .xpi extensions) for Debian source packages
>> version 3?
> Not really. First of all, that support should have been implemented with
> the rest so that it's available in lenny already. Then many options and
> features rely on the fact that tar is used. So it would probably
> requires addition of a supplementary abstraction/indirection.
Adding .zip would also be rather a pain for things like lintian. Not
impossible, but lintian assumes that it can get tar listings of Debian
packages and knows their format. Different compression methods don't
change that, but .zip is a completely different file format that requires
a completely different set of tools.
I would prefer that we not go this direction. I don't think the
additional complexity is really worth it.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
Vincent Danjean
This requirement is here to ensure smooth upgrade (stable -> testing (or new stable)
with dpkg from stable). So it stand for .bzip2 and .lzma support in *binary* packages.
Charles is talking about *sources* packages.
Some packages in testing already require other packages from testing to build
themselves (ie build-depends nor present in stable). This is catch by debbuild for
example.
Unpacking a source package is not needed during an upgrade. However, it occurs before
a build.
So, I understand the question of Charles as "do we want that stable dpkg be able to
unpack all packages from testing ?". I have no strong opinion for this.
Regards,
Vincent
> Cheers,
> Emilio
>
--
Vincent Danjean GPG key ID 0x9D025E87 vdan...@debian.org
GPG key fingerprint: FC95 08A6 854D DB48 4B9A 8A94 0BF7 7867 9D02 5E87
Unofficial pacakges: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo: deb http://perso.debian.org/~vdanjean/debian unstable main
Philipp Kern
> Unpacking a source package is not needed during an upgrade. However, it occurs before
> a build.
>
> So, I understand the question of Charles as "do we want that stable dpkg be able to
> unpack all packages from testing ?". I have no strong opinion for this.
Our infrastructure relies on this. So yes. Both dak and sbuild do this.
(At least the latter used to do it in older versions, I didn't check the
new one.)
Kind regards,
Philipp Kern
Roger Leigh
> On 2009-08-02, Vincent Danjean <vdanj...@free.fr> wrote:
> > Unpacking a source package is not needed during an upgrade. However, it occurs before
> > a build.
> >
> > So, I understand the question of Charles as "do we want that stable dpkg be able to
> > unpack all packages from testing ?". I have no strong opinion for this.
>
> Our infrastructure relies on this. So yes. Both dak and sbuild do this.
> (At least the latter used to do it in older versions, I didn't check the
> new one.)
The new sbuild always uses dpkg-source from inside the build environment,
so there's no restriction there.
Regards,
Roger
--
.''`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
Goswin von Brederlow
> Another question that I would like to ask is on the auto-patching
> functionality. One of the programs we package, EMBOSS, is released once a year
> every 15th of July, and other updates are made via patches. Currently it is
> possible to just give the patch to quilt to apply it. With the new source
> format ‘3.0 (quilt)’, it is not possible. Do you think it would be possible to
> change this behavior or at least to disable the auto-patching facilities? It
> would be of course easy to convert the patch, but I would really prefer to stay
> as identical to upstream's materials as possible.
>
> Have a nice sunday,
Why not just use 3.0 (native) format then?
MfG
Goswin
Giacomo A. Catenazzi
> Emilio Pozuelo Monfort wrote:
>> Charles Plessy wrote:
>>> I see that .bzip2 and .lzma are also supported compression methods for the 3.0
>>> (native) format as well as for the binary packages. But I do not think it would
>>> be useful to add zip to this list. It seems to me that the only thing needed is
>>> the capacity to unpack the original upstream sources. In that case there would
>>> not be a need for a Lenny support, isn't it?
>> You need it to be supported in stable before using it in unstable. So at best,
>> you would need to implement it in Squeeze and wait for Squeeze+1 to use it.
>
> This requirement is here to ensure smooth upgrade (stable -> testing (or new stable)
> with dpkg from stable). So it stand for .bzip2 and .lzma support in *binary* packages.
>
> Charles is talking about *sources* packages.
> Some packages in testing already require other packages from testing to build
> themselves (ie build-depends nor present in stable). This is catch by debbuild for
> example.
>
> Unpacking a source package is not needed during an upgrade. However, it occurs before
> a build.
>
> So, I understand the question of Charles as "do we want that stable dpkg be able to
> unpack all packages from testing ?". I have no strong opinion for this.
In stable servers I use sometime the sources of unstable/testing and I backport them.
So I would like to have package compatibility "actual stable" to "actual unstable".
I do this things only on small packages, so I don't want to upgrade/backport dpkg.
Anyway such things are not a common usage which should be supported, so I can
live also with a breaking source package compatibility.
ciao
cate
Raphael Hertzog
> I see that .bzip2 and .lzma are also supported compression methods for the 3.0
> (native) format as well as for the binary packages. But I do not think it would
> be useful to add zip to this list. It seems to me that the only thing needed is
> the capacity to unpack the original upstream sources. In that case there would
> not be a need for a Lenny support, isn't it?
As stated elsewhere, its's good to be able to unpack all source packages
with the current stable dpkg-source.
> The patch is not in unified format, which causes the failure of
> dpkg-buildpackage. It is trivial to refresh it with quilt to the unified
> format, but this introduces a divergence with upstream that I would prefer to
> avoid, because it makes it difficult for others to verify that we did not
> modify it. Can you support in the format ’3.0 (quilt)’ patches that are
> accepted by default quilt installation?
No. I really prefer that we uniformize the patches that we provide through
debian/patches/ as it's an external interface as well (and for people reviewing
patches, unified format is certainly the most common format).
Furthermore it would require disabling quite a lot of checks that are
currently in place.
Cheers,
--
Raphaël Hertzog
Charles Plessy
>
> > The patch is not in unified format, which causes the failure of
> > dpkg-buildpackage. It is trivial to refresh it with quilt to the unified
> > format, but this introduces a divergence with upstream that I would prefer to
> > avoid, because it makes it difficult for others to verify that we did not
> > modify it. Can you support in the format ’3.0 (quilt)’ patches that are
> > accepted by default quilt installation?
>
> No. I really prefer that we uniformize the patches that we provide through
> debian/patches/ as it's an external interface as well (and for people reviewing
> patches, unified format is certainly the most common format).
If this patch is to be reviewed, then in my opinion it is better to keep it in
the original upstream format, because the first thing to check would be to make
sure that it is not altered nor obsolete. Nobody ever complained about patches
not in unified format in debian/patches before on this list. Lastly, let me
stress out again that this is an upstream patch. Reviewing it has as much
relevance as reviewing a couple of source files taken randomly in the original
source archive.
If using the unified format is to become a “must” in debian/patches, I would
prefer to have it discussed rather than imposed silently. I would understand
that your time to develop support for non-unified format is limited, but if the
reason is that you want to enforce one format, then as a maintainer I think
that I should have my word to say on how I manage my packages.
In the meantime, I will move the patches from debian/patches to debian/patch,
so that it does not block the switch to the format 3.0, which brings some
improvements that are much welcome in addition to the patch management, that I
think goes in the wrong direction.
Have a nice day,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Paul Wise
unified diffs for updates, tarballs for every release or a git/etc
repository?
--
bye,
pabs
http://wiki.debian.org/PaulWise
Charles Plessy
> Perhaps you could talk to upstream about switching to either using
> unified diffs for updates, tarballs for every release or a git/etc
> repository?
For sure, Debian can suggest them git, Ubuntu can suggest them bzr, Fedora can
suggest them cvs, and Opensuze can suggest them svn.
I do not mind working with patches instad of archive updates. For Debian, it
also has the advantage of not having 12 orig.gz updates per year (12 × 20 Mo
archives). I do not know how it matters with recent hard drives, however…
And for the format of the patch, I do not know what to tell them apart that
unified diff is the preferred format of some Debian developers, and that we
like that others use the formats that we prefer. I think that is a too weak
argument, so unless there is a real flaw in the format used upstream, I will
not bother them for a change. This formats works with quilt, so I do not
understand why it would be difficult to get it work with the format ’3.0
(quilt)’ of dpkg. According to the current discussion, the problem is more
political than technical.
We already do not manage to agree internally on what is the best workflow. I
can propose Upstream to adapt their habits to our habits, but this has to come
with benefits that overcome the energy invested in the changes, and the fact
that what is best for distributor A will not match what distributor B expects.
Much saner in my opinion is to have a toolchain that is liberal in what it
accepts. (Hence the proposition to accept upstream ‘zip’ archives).
Have a nice day,
--
Charles Plessy
Tsurumi, Kanagawa, Japan
Ben Finney
> Le Tue, Aug 04, 2009 at 09:51:26AM +0200, Paul Wise a écrit :
> > Perhaps you could talk to upstream about switching to either using
> > unified diffs for updates, tarballs for every release or a git/etc
> > repository?
>
> For sure, Debian can suggest them git, Ubuntu can suggest them bzr,
> Fedora can suggest them cvs, and Opensuze can suggest them svn.
Fine. Whichever one of those they choose, it can consume and produce
unified-diff-format patches.
> And for the format of the patch, I do not know what to tell them apart
> that unified diff is the preferred format of some Debian developers,
That's quite a misrepresentation; it's far wider than just “some Debian
developers” who prefer that format.
> and that we like that others use the formats that we prefer.
The point, rather, seems to be that unified-diff format is the de facto
standard format for exchanging patch information.
> I think that is a too weak argument, so unless there is a real flaw in
> the format used upstream, I will not bother them for a change.
The flaw is that patch information in any format other than unified-diff
format is nowhere near as portable.
> Much saner in my opinion is to have a toolchain that is liberal in
> what it accepts. (Hence the proposition to accept upstream ‘zip’
> archives).
This is in opposition to the ideal of having standard [0] formats for
data interchange, and choosing them on the basis of what is already
widely produced and accepted.
[0] “standard” in this usage necessarily including “freely-implemented”,
which doesn't disqualify the other options being discussed, but I
put this footnote in the hope of forestalling useless discussions
about proprietary formats.
--
\ “I moved into an all-electric house. I forgot and left the |
`\ porch light on all day. When I got home the front door wouldn't |
_o__) open.” —Steven Wright |
Ben Finney
Michael Banck
> And for the format of the patch, I do not know what to tell them apart that
> unified diff is the preferred format of some Debian developers,
It's the preferred format for 99% of all Free Software work/projects
AFAICT.
> and that we like that others use the formats that we prefer. I think
> that is a too weak argument, so unless there is a real flaw in the
> format used upstream, I will not bother them for a change. This
> formats works with quilt, so I do not understand why it would be
> difficult to get it work with the format ???3.0 (quilt)??? of dpkg.
> According to the current discussion, the problem is more political
> than technical.
If all fails, you can still drop the upstream patches in
debian/upstream-patches/ or so for your own purposes.
Michael
Cyril Brulebois
> On Wed, Aug 05, 2009 at 10:15:22AM +0900, Charles Plessy wrote:
> > And for the format of the patch, I do not know what to tell them
> > apart that unified diff is the preferred format of some Debian
> > developers,
>
> It's the preferred format for 99% of all Free Software work/projects
> AFAICT.
I was wondering who could be in the last 1%. At least OpenSceneGraph
people[1] prefer being sent the whole files. :)
1. http://www.openscenegraph.org/projects/osg/wiki/MailingLists/SubmissionsProtocol
Mraw,
KiBi.
Charles Plessy
>
> The point, rather, seems to be that unified-diff format is the de facto
> standard format for exchanging patch information.
Le Wed, Aug 05, 2009 at 10:53:21AM +0200, Michael Banck a �crit :
>
> It's the preferred format for 99% of all Free Software work/projects
> AFAICT.
In my workplace's cafeteria, 99 % of the people eat curry rice with a spoon,
and 1 % with chopsticks. But this is causing no trouble, and never the spoon
users ask the chopsticks users to change their instrument (and I can tell you
that I do not leave a single grain of rice when I eat my curry rice with
chopsticks).
I am all for campaigning for the unified diff format if there are arguments on
which I can base a discussion with Upstream, but a mere cultural preference,
be it the one of a very large majority, is a too weak argument.
The only practical argument currently is the regression between the dpkg format
1 to 3.0 (quilt), where non-unified patches in debian/patches cause the source
package to be not buildable, only because the Dpkg maintainers decided to
reject non-unified patches despite they can be applied by the patch command
they use in Dpkg::Source::Patch.pm. How is that relevant for Upstream ?
After deleting the following check, my source package builds fine.
--- a/scripts/Dpkg/Source/Patch.pm
+++ b/scripts/Dpkg/Source/Patch.pm
@@ -325,9 +325,6 @@ sub analyze {
unless (defined($_ = getline($diff_handle))) {
error(_g("diff `%s' finishes in middle of ---/+++ (line %d)"), $diff, $.);
}
- unless (s/^\+\+\+ //) {
- error(_g("line after --- isn't as expected in diff `%s' (line %d)"), $diff, $.);
- }
$_ = strip_ts($_);
if ($_ eq '/dev/null' or s{^(\./)?[^/]+/}{$destdir/}) {
$fn2 = $_;
Why not just applying the patches and catch errors if there are, instead of
writing a new patch parser from scratch just to check that it looks like being
in the unified format?
Have a nice day,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Cyril Brulebois
> In my workplace's cafeteria, 99 % of the people eat curry rice with a
> spoon, and 1 % with chopsticks. But this is causing no trouble, and
> never the spoon users ask the chopsticks users to change their
> instrument (and I can tell you that I do not leave a single grain of
> rice when I eat my curry rice with chopsticks).
Should we care?
> I am all for campaigning for the unified diff format if there are
> arguments on which I can base a discussion with Upstream, but a mere
> cultural preference, be it the one of a very large majority, is a too
> weak argument.
According to a quick look at the diff wikipedia page[1], unified diffs
appeared in GNU diff 1.15, released in January 1991.
1. http://en.wikipedia.org/wiki/Diff
Time to move on?
Anyway: Heard about the thing called “context”?
Mraw,
KiBi.
Ben Finney
> Le Wed, Aug 05, 2009 at 03:45:00PM +1000, Ben Finney a écrit :
> >
> > The point, rather, seems to be that unified-diff format is the de
> > facto standard format for exchanging patch information.
>
> Le Wed, Aug 05, 2009 at 10:53:21AM +0200, Michael Banck a écrit :
> >
> > It's the preferred format for 99% of all Free Software work/projects
> > AFAICT.
>
> In my workplace's cafeteria, 99 % of the people eat curry rice with a
> spoon, and 1 % with chopsticks. But this is causing no trouble
Right, because an individual's use of spoon or chopsticks to eat their
own meal isn't about interaction *between* people; it's a private choice
that affects only that individual.
The analogy doesn't hold for this discussion, since this is about data
interchange formats, which affects *all* parties in the transaction.
See how far you'd get with expecting accommodation of 1% of people using
a different form of currency to pay for their curry rice.
> I am all for campaigning for the unified diff format if there are
> arguments on which I can base a discussion with Upstream, but a mere
> cultural preference, be it the one of a very large majority, is a too
> weak argument.
Standard data interchange formats is such an argument: one which you
even quoted me as putting forth. The de facto standard data format for
interchange of patch data is unified-diff format.
--
\ “Well, my brother says Hello. So, hooray for speech therapy.” |
`\ —Emo Philips |
_o__) |
Giacomo A. Catenazzi
> Le Wed, Aug 05, 2009 at 03:45:00PM +1000, Ben Finney a �crit :
>> The point, rather, seems to be that unified-diff format is the de facto
>> standard format for exchanging patch information.
>
>> It's the preferred format for 99% of all Free Software work/projects
>> AFAICT.
>
> In my workplace's cafeteria, 99 % of the people eat curry rice with a spoon,
> and 1 % with chopsticks. But this is causing no trouble, and never the spoon
> users ask the chopsticks users to change their instrument (and I can tell you
> that I do not leave a single grain of rice when I eat my curry rice with
> chopsticks).
>
> I am all for campaigning for the unified diff format if there are arguments on
> which I can base a discussion with Upstream, but a mere cultural preference,
> be it the one of a very large majority, is a too weak argument.
I think there is few advantages on non-unified diff:
I don't think many upstreams will take advantage of it, and I think
few upstream will take patched from our source package.
IMHO the right way to sent patched upstream is send comments and patches,
in they preferred form (format of patch, format of comment, target
(like mailing list) and last upstream version).
Note also: rediff works only on unified patches.
The disadvantages: we need to build a lot of tools to test quality
of our packages. I think handling different diff format will
decrement the quality of such tests.
I really think that in our future we will have a lot of automatic testing
tools to detect problem before they happens.
ciao
cate
Tollef Fog Heen
| I am all for campaigning for the unified diff format if there are
| arguments on which I can base a discussion with Upstream, but a mere
| cultural preference, be it the one of a very large majority, is a too
| weak argument.
They're easier to review (because you have a bit of context) and to
adapt to sources where they don't apply 100%.
I would start by asking upstream first and see if they reject the
request, or ask «why?» before pushing for changes in Debian tools.
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
Charles Plessy
> According to a quick look at the diff wikipedia page[1], unified diffs
> appeared in GNU diff 1.15, released in January 1991.
>
> 1. http://en.wikipedia.org/wiki/Diff
>
> Time to move on?
Le Wed, Aug 05, 2009 at 03:22:14PM +0200, Giacomo A. Catenazzi a �crit :
>
> The disadvantages: we need to build a lot of tools to test quality
> of our packages. I think handling different diff format will
> decrement the quality of such tests.
Le Wed, Aug 05, 2009 at 06:56:05PM +0200, Tollef Fog Heen a �crit :
>
> They're easier to review (because you have a bit of context) and to
> adapt to sources where they don't apply 100%.
Le Wed, Aug 05, 2009 at 11:23:11PM +1000, Ben Finney a �crit :
>
> Standard data interchange formats is such an argument: one which you
> even quoted me as putting forth. The de facto standard data format for
> interchange of patch data is unified-diff format.
So to summarise, you are suggesting me to write upstream that:
1) We want to review their patches,
2) We can not do this with context diffs,
3) We do want to actively reject non-unified diffs despite our tools work well with them,
4) The reason why they should adopt a new diff format is because it is new.
May I have some evidence that somebody really wants to review their patch? As I
explained already, it is not a random patch grabbed from their BTS, it is their
standard way to publish official corrections that change a few lines in an
archive of 20 Mo. I do not think there is more reason to review this patch than
any other change that they make when they release a new version.
What is next? Will the Project decide a standard whitespace policy and nitpick
every upstream project that does not respect it?
The only patch review system I know in Debian works well with context diffs
(http://patch-tracking.debian.net/package/emboss/6.1.0-2). Quilt, patch, diffstat,
all work well with context diff. dpkg-dev itself works well with context diffs.
The only reason it fails is that there is a political decision to reject
non-unified diffs.
I have moved the patches from debian/patches to debian/patch, which circumvents
the problem, since there is no will to compromise on either side.
Have a nice day,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Cyril Brulebois
> But, since we tweak the headers, the check can get added before the
> first dereferencement. Of course, there are the fuzzy stuff with patch,
> but sounds less likely to happen.
Heh, might have left that to the attentive reader, but let's fix the
typo: s/before/after/. :)
Mraw,
KiBi.
Cyril Brulebois
> So to summarise, you are suggesting me to write upstream that:
>
> 1) We want to review their patches,
> 2) We can not do this with context diffs,
> 3) We do want to actively reject non-unified diffs despite our tools work well with them,
Sure.
> 4) The reason why they should adopt a new diff format is because it is new.
It *was*. 20 years ago.
> May I have some evidence that somebody really wants to review their
> patch?
Any NMUer, any QA folks, any security-team guy, etc. Guess what? A
package can have a lot of eyes going over it, that don't belong to the
Maintainer/Uploaders.
> As I explained already, it is not a random patch grabbed from their
> BTS, it is their standard way to publish official corrections that
> change a few lines in an archive of 20 Mo.
AHAHAH. Want to talk about amule again? If that was still needed, that's
a *very* good reason to check what upstream does, especially when it
comes to security. And having *context* when it comes to a *20MB* (you
used the French unit, FWIW, just to clarify to other readers) *might* be
a very good idea.
> What is next? Will the Project decide a standard whitespace policy and
> nitpick every upstream project that does not respect it?
Delirium Tremens? Unified diffs are not about whitespace-only changes,
as you were already explained.
> The only patch review system I know in Debian works well with context
> diffs (http://patch-tracking.debian.net/package/emboss/6.1.0-2).
> Quilt, patch, diffstat, all work well with context diff. dpkg-dev
> itself works well with context diffs.
You probably don't want to list each and every stuff for which we keep
some sort of backward-compatibility. Or you should probably get started.
> The only reason it fails is that there is a political decision to
> reject non-unified diffs.
Sure.
Oh, if you really need an example, what about the following? We tend to
fix GCC issues. We tweak headers. Some might get added, some might be
removed. We have such a patch. A CVE arrives. A context diff gets
published. It gets applies on the top of the other patches. Say it's
something like:
| > BUGON(my_pointer_shall_not_be_null);
But, since we tweak the headers, the check can get added before the
first dereferencement. Of course, there are the fuzzy stuff with patch,
but sounds less likely to happen.
Of course, that's only about politics and victimization.
Mraw,
KiBi.
Cyril Brulebois
> Heh, might have left that to the attentive reader, but let's fix the
> typo: s/before/after/. :)
And given I wasn't even using the right option, I'm going to hide for a
while, lalala. (Thanks pusling.)
Mraw,
KiBi.
Paul Wise
> Charles Plessy <ple...@debian.org> (06/08/2009):
>> 4) The reason why they should adopt a new diff format is because it is new.
>
> It *was*. 20 years ago.
Perhaps it is about time it was made the default :)
--
bye,
pabs
http://wiki.debian.org/PaulWise
Michael Banck
> So to summarise, you are suggesting me to write upstream that:
Why do you need to write anything to upstream? If they still insist on
context diffs (or whatever that is called), I guess there is not much we
can do. Debian insists (for good reasons) on unified diffs. You can of
course ask them to reconsider again, but (as pointed out numerous times)
there is no reason to bring up anything like "the dpkg maintainer
oppresses you", it's a simle fact that unified diffs are the de-facto
standard in the FLOSS world.
The sensible way is to work around upstream by transforming their diffs
into unified diffs and storing the original diff in the source package
as well for reference. Another option would be to make up a new orig
tarball (a bit more hassle, but certainly feasable).
The other alternative is to just make up your own source package patch
system and use 3.0-native as Goswin suggested (though I have no clue how
that works).
Michael
Ben Pfaff
> Oh, if you really need an example, what about the following? We tend to
> fix GCC issues. We tweak headers. Some might get added, some might be
> removed. We have such a patch. A CVE arrives. A context diff gets
> published. It gets applies on the top of the other patches. Say it's
> something like:
> | > BUGON(my_pointer_shall_not_be_null);
>
> But, since we tweak the headers, the check can get added before the
> first dereferencement. Of course, there are the fuzzy stuff with patch,
> but sounds less likely to happen.
If you are going to argue against diffs that do not have any
context, please say so. Do not confuse the issue by instead
arguing against "context diffs", because context diffs and
unified diffs have exactly the same properties, just different
formatting.
Here is the diff manual's introduction to showing context:
Usually, when you are looking at the differences between files, you will
also want to see the parts of the files near the lines that differ, to
help you understand exactly what has changed. These nearby parts of the
files are called the "context".
GNU `diff' provides two output formats that show context around the
differing lines: "context format" and "unified format". It can
optionally show in which function or section of the file the differing
lines are found.
Here is the example of a context diff from the "diff" manual:
*** lao Sat Jan 26 23:30:39 1991
--- tzu Sat Jan 26 23:30:50 1991
***************
*** 1,7 ****
- The Way that can be told of is not the eternal Way;
- The name that can be named is not the eternal name.
The Nameless is the origin of Heaven and Earth;
! The Named is the mother of all things.
Therefore let there always be non-being,
so we may see their subtlety,
And let there always be being,
--- 1,6 ----
The Nameless is the origin of Heaven and Earth;
! The named is the mother of all things.
!
Therefore let there always be non-being,
so we may see their subtlety,
And let there always be being,
***************
*** 9,11 ****
--- 8,13 ----
The two are the same,
But after they are produced,
they have different names.
+ They both may be called deep and profound.
+ Deeper and more profound,
+ The door of all subtleties!
Here is the diff manual's comparison between context diffs and
unified diffs:
The unified output format is a variation on the context format that is
more compact because it omits redundant context lines. To select this
output format, use the `-U LINES', `--unified[=LINES]', or `-u' option.
The argument LINES is the number of lines of context to show. When it
is not given, it defaults to three.
At present, only GNU `diff' can produce this format and only GNU
`patch' can automatically apply diffs in this format. For proper
operation, `patch' typically needs at least two lines of context.
--
Ben Pfaff
http://benpfaff.org
Russ Allbery
diffs can be trivially converted into each other? They have the same
amount of information.
filterdiff --format=unified < context.diff
filterdiff --format=context < unified.diff
(filterdiff comes with patchutils.) Given that, this seems like a tempest
in a teapot to me. Just convert the diff into whatever format the tool
that you're using expects or the reviewer wants to read.
diff -c diffs are somewhat easier to read in some specific circumstances,
usually involving significant code rewrites, than diff -u diffs.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
Pierre Habouzit
> Those are actually valid ed scripts IIRC.
Okay, sorry, I meant to remove that sentence that is actually wrong...
sorry 'bout that.
--
·O· Pierre Habouzit
··O madc...@debian.org
OOO http://www.madism.org
Pierre Habouzit
> After deleting the following check, my source package builds fine.
>
> --- a/scripts/Dpkg/Source/Patch.pm
> +++ b/scripts/Dpkg/Source/Patch.pm
> @@ -325,9 +325,6 @@ sub analyze {
> unless (defined($_ = getline($diff_handle))) {
> error(_g("diff `%s' finishes in middle of ---/+++ (line %d)"), $diff, $.);
> }
> - unless (s/^\+\+\+ //) {
> - error(_g("line after --- isn't as expected in diff `%s' (line %d)"), $diff, $.);
> - }
> $_ = strip_ts($_);
> if ($_ eq '/dev/null' or s{^(\./)?[^/]+/}{$destdir/}) {
> $fn2 = $_;
>
> Why not just applying the patches and catch errors if there are, instead of
> writing a new patch parser from scratch just to check that it looks like being
> in the unified format?
FWIW I've read this sub-thread with some kind of consternation,
especially seeing how wrong some arguments are.
First of all, non-unified diffs are called "context diffs", and can have
... wait for it ... context. Those are actually valid ed scripts IIRC.
The reason why unified diffs are prefered is because those are usually
way more readable.
For example, vim upstream is using non unified diffs with context as a
way to release its incremental versions, still nowadays.
That said, yes, using non-unified diff is as laughable as using RCS or
SCCS nowadays. Though I consider it a bug if dpkg refuses to apply a
patch that patch(1) (that it uses in the end) would apply fine. I shall
say that I absolutely don't get why there even is an "analyze()" routine
in Patch.pm, if you want that, let patch(1) do it using its --dry-run
mode. It'll report failed hunk and bad syntax the same way.
I concur with Charles that it's a dpkg-dev bug, I don't think his patch
is valid though. I'm unsure if it needed all that trolling, and that a
minor bug on dpkg-dev asking for non-unified diffs support is in order.
In between there are as many already said many workarounds, the easier
one being to regenerate the diff, which you can easily do using
combinediff (I've not checked if there is anything more
straightforward):
$ cat aa
toto
tuti
titi
$ cat ab
toto
tutu
titi
$ diff -C2 aa ab | tee aa-to-ab.diff
*** aa 2009-08-06 18:25:44.875327948 +0200
--- ab 2009-08-06 18:25:50.107327652 +0200
***************
*** 1,3 ****
toto
! tuti
titi
--- 1,3 ----
toto
! tutu
titi
$ combinediff aa-to-ab.diff /dev/null
unchanged:
--- aa 2009-08-06 18:25:44.875327948 +0200
+++ ab 2009-08-06 18:25:50.107327652 +0200
@@ -1,3 +1,3 @@
toto
-tuti
+tutu
titi
Cheers,
--
Intersec <http://www.intersec.com>
Pierre Habouzit <pierre....@intersec.com>
Tél : +33 (0)1 5570 3346
Mob : +33 (0)6 1636 8131
Fax : +33 (0)1 5570 3332
37 Rue Pierre Lhomme
92400 Courbevoie
Russ Allbery
> FWIW I've read this sub-thread with some kind of consternation,
> especially seeing how wrong some arguments are.
>
> First of all, non-unified diffs are called "context diffs", and can have
> ... wait for it ... context. Those are actually valid ed scripts IIRC.
> The reason why unified diffs are prefered is because those are usually
> way more readable.
I think you're confusing the many different non-unified diff formats.
There are at least four.
1. Classic diff format (the default)
3c3
< baz
---
> bar
2. ed script format (diff -e)
3c
bar
.
3. RCS diff format (diff -n)
d3 1
a3 1
bar
4. Classic context diff format (diff -c)
*** foo.1 2009-08-06 09:38:26.000000000 -0700
--- foo.2 2009-08-06 09:38:35.000000000 -0700
***************
*** 1,6 ****
foo
bar
! baz
foo
bar
baz
--- 1,6 ----
foo
bar
! bar
foo
bar
baz
The fourth format is entirely equivalent, in informational content, to a
unified diff, just formatted differently. That form and unified diff form
are mutually convertible.
Unified diff is generally considered to be easier to read for most use
cases, but can be nigh-incomprehensible when there are lots of changes in
the same area of the code since it tries a bit too hard to find
similarities and tends to think that some blank line or brace in the
middle of a changed block is the same line between the two versions. In
those cases, classic context diff is easier to read.
The first three formats have no context, are extremely fragile if anything
changes about the file to which one is applying the diff (*particularly*
two and three), and should basically never be used for anything any more.
The fourth style above is exactly equivalent to a unified diff in terms of
information content and is really just a matter of taste.
> I concur with Charles that it's a dpkg-dev bug,
Likewise.
> I don't think his patch is valid though. I'm unsure if it needed all
> that trolling, and that a minor bug on dpkg-dev asking for non-unified
> diffs support is in order.
I think dpkg should accept old-format context diffs but reject all diffs
without context (including unified diffs without context). Diffs without
context are dangerous. They may apply in places where they're not
supposed to and make changes that are entirely unexpected.
--
Russ Allbery (r...@debian.org) <http://www.eyrie.org/~eagle/>
Manoj Srivastava
> So... you all realize, right, that old-style context diffs and unified
> diffs can be trivially converted into each other? They have the same
> amount of information.
>
> filterdiff --format=unified < context.diff
> filterdiff --format=context < unified.diff
>
> (filterdiff comes with patchutils.) Given that, this seems like a tempest
> in a teapot to me. Just convert the diff into whatever format the tool
> that you're using expects or the reviewer wants to read.
>
> diff -c diffs are somewhat easier to read in some specific circumstances,
> usually involving significant code rewrites, than diff -u diffs.
Also, emacs diff-mode does this in the buffer for you, so you
don't need to even have filterdiff installed, assuming you worship at
the altar of the one true editor.
manoj
--
Oh what a tangled web we weave, when first we practice to
deceive. Shakespeare
Manoj Srivastava <sriv...@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Charles Plessy
>
> (filterdiff comes with patchutils.) Given that, this seems like a tempest
> in a teapot to me. Just convert the diff into whatever format the tool
> that you're using expects or the reviewer wants to read.
Hi Russ and everybody,
I already explained that I prefered that the patch stays in the original format
so that it is easy for everybody (co-maintainers, reviewers, users) to verify
that it was not altered.
Giving a standard interface to reviewers is a laudable goal, but I do not see
reviewers except in elaborate scenarios about security. Therefore I will not
trade a real benefit for a hypothetical one, even if both are neglectible.
Also, I think that it is very important in a project of 1,000 persons to stick
to facts, and avoid building illusions together. So as long as there is no
reviewing process nor package reviews, there is no need to adapt to imaginary
reviewers.
This said, if there were a project-wide momentum for standardising on one patch
format, I would not oppose. This would probably be a release goal, a
preparation for a Policy change, or a demand from the security team to the
package manintainers. Something with a motivation, a plan, some facts and some
volunteers to get things done.
I have reassigned the bug of my package to dpkg-dev. It is holiday season, so
let's wait for a while and see the answer of the dpkg developers.
Have a nice day,
--
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan
Ben Finney
> First of all, non-unified diffs are called "context diffs"
Not necessarily. I've been using the term to reply to *any* diff output
format that isn't unified-diff format. From my perspective, there is the
de facto standard of unified-diff format, used by the vast majority for
patch data interchange; and there is the vanishing minority of any other
patch format, which should be deprecated.
> I concur with Charles that it's a dpkg-dev bug, I don't think his
> patch is valid though. I'm unsure if it needed all that trolling, and
> that a minor bug on dpkg-dev asking for non-unified diffs support is
> in order.
Rather, for specifically “context diff” format. +1 to that suggestion;
I'm fine with tools accepting it, but it should be strongly deprecated
(by social, not technical, means).
--
\ “The industrial system is profoundly dependent on commercial |
`\ television and could not exist in its present form without it.” |
_o__) —John Kenneth Galbraith, _The New Industrial State_, 1967 |
Ben Finney
Ben Finney
> Pierre Habouzit <madc...@madism.org> writes:
>
> > First of all, non-unified diffs are called "context diffs"
>
> Not necessarily. I've been using the term to reply to *any* diff output
> format that isn't unified-diff format.
s/reply to/refer to/
--
\ “One seldom discovers a true believer that is worth knowing.” |
`\ —Henry L. Mencken |
_o__) |
Pierre Habouzit
> Le Thu, Aug 06, 2009 at 09:26:02AM -0700, Russ Allbery a écrit :
> >
> > (filterdiff comes with patchutils.) Given that, this seems like a tempest
> > in a teapot to me. Just convert the diff into whatever format the tool
> > that you're using expects or the reviewer wants to read.
>
> Hi Russ and everybody,
>
> I already explained that I prefered that the patch stays in the original format
Then you'll need to write your "own" patch system that calls patch(1) to
apply the patches, à la cdbs-simple-patchsys.
--
Intersec <http://www.intersec.com>
Pierre Habouzit <pierre....@intersec.com>
Tél : +33 (0)1 5570 3346
Mob : +33 (0)6 1636 8131
Fax : +33 (0)1 5570 3332
37 Rue Pierre Lhomme
92400 Courbevoie
Neil McGovern
> reviewers except in elaborate scenarios about security. Therefore I will not
> trade a real benefit for a hypothetical one, even if both are neglectible.
> Also, I think that it is very important in a project of 1,000 persons to stick
> to facts, and avoid building illusions together. So as long as there is no
> reviewing process nor package reviews, there is no need to adapt to imaginary
> reviewers.
>
/me raises his release team hat.
Neil
--
* Tolimar votes for debconf7 to be somewhere where he speaks the
language.
<Tolimar> That would a veto for switzerland ;)
<Ganneff> Tolimar: that also vetos germany
Peter Eisentraut
> Michael Banck <mba...@debian.org> (05/08/2009):
> > On Wed, Aug 05, 2009 at 10:15:22AM +0900, Charles Plessy wrote:
> > > And for the format of the patch, I do not know what to tell them
> > > apart that unified diff is the preferred format of some Debian
> > > developers,
> >
> > It's the preferred format for 99% of all Free Software work/projects
> > AFAICT.
>
> I was wondering who could be in the last 1%. At least OpenSceneGraph
> people[1] prefer being sent the whole files. :)
For everyone's further entertainment: The PostgreSQL project heavily prefers
context diff patch submissions. This is also part of the reason why
PostgreSQL is still stuck on CVS, because Git does not produce context diffs.
There you go.
Frank Küster
> On Fri, Aug 07, 2009 at 10:45:14AM +0900, Charles Plessy wrote:
>> Le Thu, Aug 06, 2009 at 09:26:02AM -0700, Russ Allbery a écrit :
>> >
>> > (filterdiff comes with patchutils.) Given that, this seems like a tempest
>> > in a teapot to me. Just convert the diff into whatever format the tool
>> > that you're using expects or the reviewer wants to read.
>>
>> Hi Russ and everybody,
>>
>> I already explained that I prefered that the patch stays in the original format
>
> Then you'll need to write your "own" patch system that calls patch(1) to
> apply the patches, à la cdbs-simple-patchsys.
Why should he need to do that? If you'd had written "submit patches to
dpkg", I could get a meaning out of it, but here? He doesn't want to
diverge from upstream.
Regards, Frank
--
Dr. Frank Küster
Debian Developer (TeXLive)
VCD Aschaffenburg-Miltenberg, ADFC Miltenberg
B90/Grüne KV Miltenberg
Frank Küster
> On Fri, Aug 07, 2009 at 10:45:14AM +0900, Charles Plessy wrote:
>> Giving a standard interface to reviewers is a laudable goal, but I do not see
>> reviewers except in elaborate scenarios about security. Therefore I will not
>> trade a real benefit for a hypothetical one, even if both are neglectible.
>> Also, I think that it is very important in a project of 1,000 persons to stick
>> to facts, and avoid building illusions together. So as long as there is no
>> reviewing process nor package reviews, there is no need to adapt to imaginary
>> reviewers.
>>
>
> /me raises his release team hat.
During a freeze, one typically wouldn't apply the upstream patches
anyway, but just cherry-pick individual changes. In this case, it
wouldn't be a problem to create unified diffs. Or even reformat the
context diff to unified, in case the upstream patch contains nothing but
RC (and translation) fixes.
Regards, Frank
--
Dr. Frank Küster
Debian Developer (TeXLive)
VCD Aschaffenburg-Miltenberg, ADFC Miltenberg
B90/Grüne KV Miltenberg
Manoj Srivastava
> This said, if there were a project-wide momentum for standardising on one patch
> format, I would not oppose. This would probably be a release goal, a
> preparation for a Policy change, or a demand from the security team to the
> package manintainers. Something with a motivation, a plan, some facts and some
> volunteers to get things done.
Why would we want to add that to policy? It seems like that both
the context diff and the unified diffs offer the same information (ie,
the change, and the context), and that each format can be mechanically
transformed from one to the other, and that which form one uses depends
on the individual.
I find that chunks with small changed grouped close together
read better in unified format, but if the chunk is long, with lots of
smallish changes interspersed with common context, then following the
logic of the old code versus the new code is better using context diffs
(you do not have to mentally keep track of two different flows).
This is mostly subjective, I know.
So policy should never promote one personal preference over
another, given that the formats convey the same information, and can be
transformed by tools, and are equally easily applied.
I mean, really, people, we are not the Borg. Why should dpkg
care, as long as the patch applies?
manoj
not inclined to pander to the hobgoblin of small minds
--
Save yourself from the 'Gates' of hell, use Linux." -- like that
one. The_Kind @ LinuxNet
Manoj Srivastava <sriv...@debian.org> <http://www.debian.org/~srivasta/>
1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C
Pierre Habouzit
> Pierre Habouzit <madc...@madism.org> wrote:
>
> > On Fri, Aug 07, 2009 at 10:45:14AM +0900, Charles Plessy wrote:
> >> Le Thu, Aug 06, 2009 at 09:26:02AM -0700, Russ Allbery a écrit :
> >> >
> >> > (filterdiff comes with patchutils.) Given that, this seems like a tempest
> >> > in a teapot to me. Just convert the diff into whatever format the tool
> >> > that you're using expects or the reviewer wants to read.
> >>
> >> Hi Russ and everybody,
> >>
> >> I already explained that I prefered that the patch stays in the original format
> >
> > Then you'll need to write your "own" patch system that calls patch(1) to
> > apply the patches, à la cdbs-simple-patchsys.
>
> Why should he need to do that? If you'd had written "submit patches to
> dpkg", I could get a meaning out of it, but here? He doesn't want to
> diverge from upstream.
Oh boy, are you even reading... That's a workaround to wait for dpkg to
be fixed. If you're willing to fix dpkg, please go ahead, but Charles
first try is clearly _not_ a proper fix, hence for now no such patch
exists.
Raphael Hertzog
thanks
On Thu, 06 Aug 2009, Pierre Habouzit wrote:
> That said, yes, using non-unified diff is as laughable as using RCS or
> SCCS nowadays. Though I consider it a bug if dpkg refuses to apply a
> patch that patch(1) (that it uses in the end) would apply fine. I shall
> say that I absolutely don't get why there even is an "analyze()" routine
> in Patch.pm,
analyze() extracts information from the patch in order to:
- create directories that do not exist yet (patch will not do that for
you AFAIK or at least that's the assumption that the current codebase
made, it might have changed since this part of the code has been written)
- have a listing of all modified files in order to harmonize their
timestamp afterwards (required to avoid unwanted rebuilding when
patching auto-generated files and their source file)
Extracting those information require to parse the patch files, so why not
error out when the parser finds something not allowed?
So while I have no opposition to allowing patches in context format (and
not other non-contextual ones), the fix is most likely not the short one
proposed by Charles. Instead the Dpkg::Source::Patch object should have
full support of the context format at least in the analyze() part.
> if you want that, let patch(1) do it using its --dry-run
> mode. It'll report failed hunk and bad syntax the same way.
You'll be surprised how much permissive patch actually is. IIRC missing context
in unidiff is not always fatal for example (even when the numbers on the
@@ lines stipulate that there should be more lines than what there is).
Cheers,
--
Raphaï¿œl Hertzog
Pierre Habouzit
> retitle 485330 Allow context diff in debian/patches/ in 3.0 (quilt) format
> thanks
>
> On Thu, 06 Aug 2009, Pierre Habouzit wrote:
> > That said, yes, using non-unified diff is as laughable as using RCS or
> > SCCS nowadays. Though I consider it a bug if dpkg refuses to apply a
> > patch that patch(1) (that it uses in the end) would apply fine. I shall
> > say that I absolutely don't get why there even is an "analyze()" routine
> > in Patch.pm,
>
> analyze() extracts information from the patch in order to:
> - create directories that do not exist yet (patch will not do that for
> you AFAIK or at least that's the assumption that the current codebase
> made, it might have changed since this part of the code has been written)
According to the man page, and to tests I did, it does
> - have a listing of all modified files in order to harmonize their
> timestamp afterwards (required to avoid unwanted rebuilding when
> patching auto-generated files and their source file)
Can't you get that using lsdiff from patchutils ?
> Extracting those information require to parse the patch files, so why not
> error out when the parser finds something not allowed?
With that we clearly agree ;)
> So while I have no opposition to allowing patches in context format (and
> not other non-contextual ones), the fix is most likely not the short one
> proposed by Charles.
It's not.
> Instead the Dpkg::Source::Patch object should have full support of the
> context format at least in the analyze() part.
Or you may use lsdiff.
> > if you want that, let patch(1) do it using its --dry-run
> > mode. It'll report failed hunk and bad syntax the same way.
>
> You'll be surprised how much permissive patch actually is. IIRC missing context
> in unidiff is not always fatal for example (even when the numbers on the
> @@ lines stipulate that there should be more lines than what there is).
Sure, but there are some tools to verify how clean a patch is, but yeah,
you're probably right on that.
Raphael Hertzog
> > - create directories that do not exist yet (patch will not do that for
> > you AFAIK or at least that's the assumption that the current codebase
> > made, it might have changed since this part of the code has been written)
>
> According to the man page, and to tests I did, it does
So there's room for code removal, cool. Any hint since when this is the case?
> > - have a listing of all modified files in order to harmonize their
> > timestamp afterwards (required to avoid unwanted rebuilding when
> > patching auto-generated files and their source file)
>
> Can't you get that using lsdiff from patchutils ?
Probably but then I indirectly add patchutils to build-essential
because dpkg-dev needs to depend on it. I have tried to avoid many
new dependencies when refactoring the code. Otherwise I could have
probably used many CPAN modules that deal with this and much more.