Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Debian package from scratch, git to pristine-tar

997 views
Skip to first unread message

Tong Sun

unread,
Jul 7, 2018, 10:40:02 PM7/7/18
to
Hi, 


> If upstream doesn't build upstream tarballs, or you don't care about them, the simplest way is to clone upstream's repository and create a separate packaging branch in there. You will not need gbp import-orig at all with this workflow. gbp buildpackage will handle creating the upstream tarballs needed for the Debian source package.

I cannot make it happening, no matter what I tried. 
How to make gbp buildpackage create the upstream tarballs and commit the generated tarball back to the pristine-tar branch?

Here is my trial log, and you can see that nothing that I tried works:


cd shc
git checkout pristine-tar

$ grep -B1 upstream-tag ~/.gbp.conf
[DEFAULT]
upstream-tag = v%(version)s

$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Pristine-tar branch "pristine-tar" not found
gbp:info: Creating /sysvol/dg/shc/shc_3.9.6.orig.tar.gz
gbp:error: v3.9.6 is not a valid treeish

$ ls -l ../
total 4
drwxrwx--x 7 u u 4096 2018-07-07 16:20 shc/

$ ls /sysvol/dg/shc/shc_3.9.6.orig.tar.gz
ls: cannot access '/sysvol/dg/shc/shc_3.9.6.orig.tar.gz': No such file or directory

uscan --force-download

$ ls -l ../
total 128
drwxrwx--x 7 u u   4096 2018-07-07 16:20 shc/
-rw-rw---- 1 u u 122644 2018-07-07 16:40 shc-3.9.6.tar.gz
-rw-rw---- 1 u u    819 2018-07-07 16:40 shc-3.9.6.tar.gz.pgp
lrwxrwxrwx 1 u u     16 2018-07-07 16:40 shc_3.9.6.orig.tar.gz -> shc-3.9.6.tar.gz

$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Unknown compression type of - [+] add debian/ from upstream packing, assuming gzip
gbp:error: v3.9.6 is not a valid treeish

$ gbp buildpackage --git-pristine-tar                   
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Unknown compression type of - [+] add debian/ from upstream packing, assuming gzip
gbp:error: Pristine-tar couldn't verify "shc_3.9.6.orig.tar.gz": fatal: Path 'shc_3.9.6.orig.tar.gz.delta' does not exist in 'refs/heads/pristine-tar'
pristine-tar: git show refs/heads/pristine-tar:shc_3.9.6.orig.tar.gz.delta failed

$ gbp buildpackage                    
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Unknown compression type of - [+] add debian/ from upstream packing, assuming gzip
gbp:error: Pristine-tar couldn't verify "shc_3.9.6.orig.tar.gz": fatal: Path 'shc_3.9.6.orig.tar.gz.delta' does not exist in 'refs/heads/pristine-tar'
pristine-tar: git show refs/heads/pristine-tar:shc_3.9.6.orig.tar.gz.delta failed

$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag=upstream
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Unknown compression type of - [+] add debian/ from upstream packing, assuming gzip
gbp:error: upstream is not a valid treeish

$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag=pristine-tar
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:warning: Unknown compression type of - [+] add debian/ from upstream packing, assuming gzip
gbp:info: Performing the build
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building shc using existing ./shc_3.9.6.orig.tar.gz
dpkg-source: info: local changes detected, the modified files are:
 shc/.travis.yml
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/shc_3.9.6-1.diff.HzG2Wj
E: Failed to package source directory /sysvol/dg/shc/shc
gbp:error: 'sbuild --source-only-changes -s -v -A --no-clean-source' failed: it exited with 1

$ cat /tmp/shc_3.9.6-1.diff.HzG2Wj
Description: <short summary of the patch>
 TODO: Put a short summary on the line above and replace this paragraph
 with a longer explanation of this change. Complete the meta-information
 with other relevant fields (see below for details). To make it easier, the
 information below has been extracted from the changelog. Adjust it or drop
 it.
 .
 shc (3.9.6-1) trusty; urgency=medium
 .
   * Fix for infinite loop (Closes: #861180)
Author: Md Jahidul Hamid <jahidu...@yahoo.com>

---
The information above should follow the Patch Tagging Guidelines, please
checkout http://dep.debian.net/deps/dep3/ to learn about the format. Here
are templates for supplementary fields that you might want to add:

Origin: <vendor|upstream|other>, <url of original patch>
Bug: <url in upstream bugtracker>
Bug-Debian: https://bugs.debian.org/<bugnumber>
Bug-Ubuntu: https://launchpad.net/bugs/<bugnumber>
Forwarded: <no|not-needed|url proving that it has been forwarded>
Reviewed-By: <name and email of someone who approved the patch>
Last-Update: 2018-07-07

--- /dev/null
+++ shc-3.9.6/.travis.yml
@@ -0,0 +1,5 @@
+dist: trusty
+sudo: required
+language: c
+script: ./autogen.sh; ./configure && make && make check
+

please help. thx. 

Geert Stappers

unread,
Jul 8, 2018, 2:40:02 AM7/8/18
to
On Sat, Jul 07, 2018 at 10:38:51PM -0400, Tong Sun wrote:
> Hi,
>
> I have a question following
> https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
> ,
>
> > If upstream doesn't build upstream tarballs, or you don't care about
> them, the simplest way is to clone upstream's repository and create a
> separate packaging branch in there. You will not need gbp import-orig at
> all with this workflow. gbp buildpackage will handle creating the upstream
> tarballs needed for the Debian source package.
>
> I cannot make it happening, no matter what I tried.
> How to make gbp buildpackage create the upstream tarballs and commit the
> generated tarball back to the pristine-tar branch?
>
> Here is my trial log, and you can see that nothing that I tried works:

If it would be easy, everbody would do it

...
>
> *please help. thx. *


`gbp clone URL` is only intended for URL which has gbp information.

That is why `gbp clone https://github.com/neurobin/shc.git`
is a start in the wrong direction.


Direction to go: Take a detour

Do `gbp clone EXISTING_GBP_REPO` and check which branches it has.
Then work on a "fresh git repo" where you add branches.


My apology for being terse.


Groeten
Geert Stappers
--
Leven en laten leven

Shengjing Zhu

unread,
Jul 8, 2018, 2:50:01 AM7/8/18
to
On Sun, Jul 8, 2018 at 10:39 AM Tong Sun <pkgoy...@neverbox.com> wrote:
> $ grep -B1 upstream-tag ~/.gbp.conf
> [DEFAULT]
> upstream-tag = v%(version)s
>
> $ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit
> dh clean --with autotools_dev
> dh_testdir
> dh_auto_clean
> dh_autotools-dev_restoreconfig
> dh_clean
> gbp:warning: Pristine-tar branch "pristine-tar" not found
> gbp:info: Creating /sysvol/dg/shc/shc_3.9.6.orig.tar.gz
> gbp:error: v3.9.6 is not a valid treeish

Except for misunderstanding the gbp work flow. This is error is caused by

> [DEFAULT]
> upstream-tag = v%(version)s

It should be `upstream-tag = %(version)s`

--
Best regards,
Shengjing Zhu

Tong Sun

unread,
Jul 8, 2018, 9:20:02 AM7/8/18
to
On Sun, Jul 8, 2018 at 2:30 AM Geert Stappers wrote:
On Sat, Jul 07, 2018 at 10:38:51PM -0400, Tong Sun wrote:
> Hi,
>
> I have a question following
> https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html
> ,
>
> > If upstream doesn't build upstream tarballs, or you don't care about
> them, the simplest way is to clone upstream's repository and create a
> separate packaging branch in there. You will not need gbp import-orig at
> all with this workflow. gbp buildpackage will handle creating the upstream
> tarballs needed for the Debian source package.
>
> I cannot make it happening, no matter what I tried.
> How to make gbp buildpackage create the upstream tarballs and commit the
> generated tarball back to the pristine-tar branch?
>
> Here is my trial log, and you can see that nothing that I tried works:

If it would be easy, everbody would do it

Well, I think it should be -- that's why & what gbp is for. 
 
`gbp clone URL` is only intended for URL which has gbp information.

That is why `gbp clone https://github.com/neurobin/shc.git`
is a start in the wrong direction.

Direction to go:   Take a detour

Do `gbp clone EXISTING_GBP_REPO` and check which branches it has.
Then work on a "fresh git repo" where you add branches.

Thanks for your input Groeten, but I'm afraid that your workflow is still pre-gbp, i.e., you are still doing steps manually that gbp is already doing it for us. Let me quota again what gbp can automatically do:

> If upstream doesn't build upstream tarballs, or you don't care about
> them, the simplest way is to clone upstream's repository and create a
> separate packaging branch in there. You will not need gbp import-orig at
> all with this workflow. gbp buildpackage will handle creating the upstream
> tarballs needed for the Debian source package.

I.e., it will do it, provided that I can find the correct way of doing it...

> `gbp clone URL` is only intended for URL which has gbp information.

That's what the limitation of `git clone URL` is, not `gbp clone`'s I'm afraid. 

Thanks for helping though. 

Tong Sun

unread,
Jul 8, 2018, 10:10:01 AM7/8/18
to
On Sat, Jul 7, 2018 at 10:38 PM Tong Sun wrote:
Hi, 


> If upstream doesn't build upstream tarballs, or you don't care about them, the simplest way is to clone upstream's repository and create a separate packaging branch in there. You will not need gbp import-orig at all with this workflow. gbp buildpackage will handle creating the upstream tarballs needed for the Debian source package.

I cannot make it happening, no matter what I tried. 
How to make gbp buildpackage create the upstream tarballs and commit the generated tarball back to the pristine-tar branch?

Thanks to Shengjing Zhu's help, I was able to move one step further. 
(Oddly I don't have it in my inbox, but found it at

I.e., what works is, 

gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s'

Which is, 

using pristine-tar™ to make gbp buildpackage commit the generated tarball back to the pristine-tar branch using the --git-pristine-tar-commit option ... This will make sure others building your package can exactly regenerate the tarball you created when building the Debian™ package.  
- - https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.import.upstream-git.html

I.e., I don't have a pristine tarball, and am using the above command to generate it for me. 

However, `gbp --git-pristine-tar` generated tarball confuses dpkg-source. Here is the full log, right from scratch:

------------------------------------

cd shc
git checkout pristine-tar

# prepare my debian/ folder, then

$ ls -l ../
total 124
drwxrwx--x 7 u u   4096 2018-07-07 16:20 shc

$ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s'
dh clean  --with autotools_dev
   dh_testdir
   dh_auto_clean
   dh_autotools-dev_restoreconfig
   dh_clean
gbp:info: Creating /sysvol/dg/shc/shc_3.9.6.orig.tar.gz
gbp:info: Performing the build
dpkg-source: info: using source format '3.0 (quilt)'
dpkg-source: info: building shc using existing ./shc_3.9.6.orig.tar.gz
dpkg-source: info: local changes detected, the modified files are:
 shc/.travis.yml
dpkg-source: info: you can integrate the local changes with dpkg-source --commit
dpkg-source: error: aborting due to unexpected upstream changes, see /tmp/shc_3.9.6-1.diff.kENb1n
E: Failed to package source directory /sysvol/dg/shc/shc
gbp:error: 'sbuild --source-only-changes -s -v -A --no-clean-source' failed: it exited with 1

$ cat /tmp/shc_3.9.6-1.diff.kENb1n
...
--- /dev/null
+++ shc-3.9.6/.travis.yml
@@ -0,0 +1,5 @@
...

$ ls -l ../
total 124
drwxrwx--x 7 u u   4096 2018-07-07 16:20 shc
-rw-rw---- 1 u u 122644 2018-07-08 09:49 shc_3.9.6.orig.tar.gz
------------------------------------

I.e., `gbp --git-pristine-tar` is able to generate the tarball, but that  tarballconfuses dpkg-source into thinking there is unexpected upstream changes -- the .travis.yml file is not in the tarball, but only in upstream git (therefore in my master as well)

------------------------------------
$ ls .travis.yml 
.travis.yml

$ tar -tvzf ../shc_3.9.6.orig.tar.gz | grep travis | wc
      0       0       0

$ grep filter ~/.gbp.conf | wc 
      0       0       0
------------------------------------

I don't have import-orig filter myself, so removing the .travis.yml file is done by `gbp --git-pristine-tar`. This is a good thing, especially when the upstream contains files like,

    '*egg.info',
    '.bzr',
    '.hg',
    '.hgtags',
    '.svn',
    'CVS',

However, then, how to make `dpkg-source` happy as well, without complaining? I shouldn't remove them from  upstream git (therefore in my master as well), should I?

Thx

Andrey Rahmatullin

unread,
Jul 8, 2018, 1:30:02 PM7/8/18
to
On Sun, Jul 08, 2018 at 10:03:53AM -0400, Tong Sun wrote:
> Thanks to Shengjing Zhu's help, I was able to move one step further.
> (Oddly I don't have it in my inbox
Are you subscribed to the list?

> I.e., what works is,
>
> gbp buildpackage --git-pristine-tar --git-pristine-tar-commit
> --git-upstream-tag='*%(version)s*'
--git-pristine-tar and --git-upstream-tag should ne set in
debian/gbp.conf, but yes.

> I don't have import-orig filter myself, so removing the .travis.yml file is
> done by `gbp --git-pristine-tar`.
No, it's done by git-archive via .gitattributes.

So it looks you need to remove them from your debian branch as well or to
find a way for gbp to ignore export-ignore in .gitattributes.

--
WBR, wRAR
signature.asc

Shengjing Zhu

unread,
Jul 8, 2018, 2:30:02 PM7/8/18
to
On Sun, Jul 8, 2018 at 10:04 PM Tong Sun <pkgoy...@neverbox.com> wrote:
> gbp clone https://github.com/neurobin/shc.git
>
> cd shc
> git checkout pristine-tar
[...]
> $ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s'

I'm not sure how these command could succeed. At least with my test,
either pristine-tar branch is not auto created, nor pristine-tar can
perform commit.
The upstream repo has same branch/tag name, which confuses pristine-tar .

> However, then, how to make `dpkg-source` happy as well, without complaining? I shouldn't remove them from upstream git (therefore in my master as well), should I?
>

That's because the upstream repo has .gitattributes, which influences
the result of `git-archive`. You can use `gbp buildpackage export-dir`
feature to let gbp do same `git-archive` for your debian branch.

Tong Sun

unread,
Jul 8, 2018, 6:20:03 PM7/8/18
to
Thanks a lot Shengjing, you've been most helpful -- always bang on, always. 


> However, then, how to make `dpkg-source` happy as well, without complaining? I shouldn't remove them from  upstream git (therefore in my master as well), should I?
>
That's because the upstream repo has .gitattributes, which influences
the result of `git-archive`. You can use `gbp buildpackage export-dir`
feature to let gbp do same `git-archive` for your debian branch.

YEP! That did the trick. THX a lot!

> gbp clone https://github.com/neurobin/shc.git
>
> cd shc
> git checkout pristine-tar
[...]
> $ gbp buildpackage --git-pristine-tar --git-pristine-tar-commit --git-upstream-tag='%(version)s'

I'm not sure how these command could succeed. At least with my test,
either pristine-tar branch is not auto created, nor pristine-tar can
perform commit.
The upstream repo has same branch/tag name, which confuses pristine-tar .

I don't know either, but it just magically works for me. With your above trick, now everything is working fine for me. I swear that what I posted in OP is all that I did. 
Did you create `pristine-tar` manually before calling `gbp buildpackage`? as in:

------
git branch pristine-tar

$ git branch -v
* master       9109084 [ahead 1] - [+] add debian/ from upstream packing
  pristine-tar 9109084 - [+] add debian/ from upstream packing
  release      379c501 release after fix #38
------

I also did, `uscan --force-download` once, but believe it is irrelevant, as I've deleted what downloaded after your fist tip. 

Besides these two, I'm absolutely sure there is no more magic that I did. 

Thanks again for all your helps!

Oh, PS, my pristine-tar does looks a bit weird to me:

------
$ git checkout pristine-tar
Switched to branch 'pristine-tar'

$ ls -1 
AUTHORS
COPYING
ChangeLog
INSTALL
Makefile.am
Makefile.in
NEWS
README
README.md
aclocal.m4
autogen.sh
config
configure
debian
man.html
man.md
shc.1
shc_3.9.6.orig.tar.gz.delta
src
test
------

I.e., besides the shc_3.9.6.orig.tar.gz.delta & shc_3.9.6.orig.tar.gz.id files, all files from my master are in there as well. I have tried to remove them all, but found that files in my master branch get deleted too. Weird. 

Andrey Rahmatullin

unread,
Jul 9, 2018, 3:00:02 AM7/9/18
to
On Sun, Jul 08, 2018 at 06:16:47PM -0400, Tong Sun wrote:
> Did you create `pristine-tar` manually before calling `gbp buildpackage`?
This is wrong.

> I.e., besides the shc_3.9.6.orig.tar.gz.delta & shc_3.9.6.orig.tar.gz.id
> files, all files from my master are in there as well.
Sure, you created it from a branch with those files, why would they
disappear?

> I have tried to remove them all, but found that files in my master
> branch get deleted too.
I don't think so.

--
WBR, wRAR
signature.asc

Tong Sun

unread,
Jul 9, 2018, 9:30:02 AM7/9/18
to
On Mon, Jul 9, 2018 at 2:53 AM Andrey Rahmatullin wrote:
>
> On Sun, Jul 08, 2018 at 06:16:47PM -0400, Tong Sun wrote:
> > Did you create `pristine-tar` manually before calling `gbp buildpackage`?
> This is wrong.

So would you explain what steps are correct instead then, to get
upstream & pristine-tar branches from alien git from scratch, please?

Andrey Rahmatullin

unread,
Jul 9, 2018, 9:40:03 AM7/9/18
to
On Mon, Jul 09, 2018 at 09:25:47AM -0400, Tong Sun wrote:
> > > Did you create `pristine-tar` manually before calling `gbp buildpackage`?
> > This is wrong.
>
> So would you explain what steps are correct instead then, to get
> upstream & pristine-tar branches from alien git from scratch, please?
I've already pointed you to the relevant gbp docs section. If you are
going to use some upstream branch as the gbp upstream-branch directly you
should just edit debian/gbp.conf. The pristine-tar branch should be
created automatically when pristine-tar is first used by gbp.

--
WBR, wRAR
signature.asc

Tong Sun

unread,
Jul 17, 2018, 3:20:02 PM7/17/18
to
On Sun, Jul 8, 2018 at 9:09 AM Tong Sun wrote:

> On Sun, Jul 8, 2018 at 2:30 AM Geert Stappers wrote:

>> . . .
>> `gbp clone URL` is only intended for URL which has gbp information.
>>
>> Direction to go: Take a detour
>>
>> Do `gbp clone EXISTING_GBP_REPO` and check which branches it has.
>> Then work on a "fresh git repo" where you add branches.
>
> Thanks for your input Groeten, but I'm afraid that your workflow is still pre-gbp, i.e., you are still doing steps manually that gbp is already doing it for us. Let me quota again what gbp can automatically do:
>
> > If upstream doesn't build upstream tarballs, or you don't care about
> > them, the simplest way is to clone upstream's repository and create a
> > separate packaging branch in there. You will not need gbp import-orig at
> > all with this workflow. gbp buildpackage will handle creating the upstream
> > tarballs needed for the Debian source package.
>
> I.e., it will do it, provided that I can find the correct way of doing it...

I'm sorry Geert,

Not only I copied your name wrong, but my interpretation about the
above paragraph is totally wrong too.

You are totally right and I was totally wrong. I don't know what
possessed me, turning me into a total dick.

Sorry about that and thanks again for your kind help

Geert Stappers

unread,
Jul 18, 2018, 5:10:02 PM7/18/18
to
Completely accepted.
No hard feelings.

Enjoy your time with Debian.
signature.asc
0 new messages