Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bug#1026230: debhelper: Unable to successfully build in read-only source directory.
73 views
Skip to first unread message
Garrett Kajmowicz
Dec 16, 2022, 1:20:03 PM12/16/22
to
Package: debhelper
Version: 13.6ubuntu1
Severity: wishlist
Dear Maintainer,
I'm working on a use-case which can be generalized as wanting to be able to
perform a build with debhelper in a read-only source directory. Not simply
that the files themselves are read-only, but that the full directory
tree is read-only. Think of building source from a directory on a CD-ROM,
a NFS mount with read-only permissions, or in my case a Docker container with
the source code on a read-only bind mount.
Many of the required capabilities are already supported. For example, the
--builddirectory flag allows the building step to be placed in a different
directory. The autotools like automake and autoconf support building
out-of-tree. However, I'm left with debhelper and related scripts themselves.
The debhelper scripts are causing a few issues. The immediate issue which has
led me to file this ticket is that debhelper really wants to write log files
into the source directory. This results in an error such as:
dh_autoreconf_clean: error: failed to write to debian/<package>.debhelper.log: Read-only file system
I tried to work around this by hacking up the debhelper scripts in the Docker
image by doing:
RUN sed -i -e 's!"debian/${ext}debhelper.log"!"/working/debian/${ext}debhelper.log"!g' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/share/perl5/Debian/Debhelper/SequencerUtil.pm
RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/bin/dh_clean
This failed because the script was unable to find the log file to delete, with
error message:
dh_autoreconf_clean: error: failed to write to /working/debian/<package>.debhelper.log: No such file or directory
The changes required are clearly more extensive than a little string
substitution. And if the work involved is going to be extensive, it should
probably be handled by someone with knowledge of (and the wisdom to) alter the
architecture as well.
One other possibly-related item of note (I can file a separate bug if you like)
and which may have an existing solution:
Passing in --buildinfo-option=-u/out still results in a -O option which
refers to the wrong directory. That is, I get the following:
dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
This resulted from an experiment where I was copying the source code to a
temporary directory with read/write permissions, but where the parent
directory was not writable.
Thank you for your time,
- Garrett
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
Version: 13.6ubuntu1
Severity: wishlist
Dear Maintainer,
I'm working on a use-case which can be generalized as wanting to be able to
perform a build with debhelper in a read-only source directory. Not simply
that the files themselves are read-only, but that the full directory
tree is read-only. Think of building source from a directory on a CD-ROM,
a NFS mount with read-only permissions, or in my case a Docker container with
the source code on a read-only bind mount.
Many of the required capabilities are already supported. For example, the
--builddirectory flag allows the building step to be placed in a different
directory. The autotools like automake and autoconf support building
out-of-tree. However, I'm left with debhelper and related scripts themselves.
The debhelper scripts are causing a few issues. The immediate issue which has
led me to file this ticket is that debhelper really wants to write log files
into the source directory. This results in an error such as:
dh_autoreconf_clean: error: failed to write to debian/<package>.debhelper.log: Read-only file system
I tried to work around this by hacking up the debhelper scripts in the Docker
image by doing:
RUN sed -i -e 's!"debian/${ext}debhelper.log"!"/working/debian/${ext}debhelper.log"!g' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/share/perl5/Debian/Debhelper/SequencerUtil.pm
RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/bin/dh_clean
This failed because the script was unable to find the log file to delete, with
error message:
dh_autoreconf_clean: error: failed to write to /working/debian/<package>.debhelper.log: No such file or directory
The changes required are clearly more extensive than a little string
substitution. And if the work involved is going to be extensive, it should
probably be handled by someone with knowledge of (and the wisdom to) alter the
architecture as well.
One other possibly-related item of note (I can file a separate bug if you like)
and which may have an existing solution:
Passing in --buildinfo-option=-u/out still results in a -O option which
refers to the wrong directory. That is, I get the following:
dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
This resulted from an experiment where I was copying the source code to a
temporary directory with read/write permissions, but where the parent
directory was not writable.
Thank you for your time,
- Garrett
*** Reporter, please consider answering these questions, where appropriate ***
* What led up to the situation?
* What exactly did you do (or not do) that was effective (or
ineffective)?
* What was the outcome of this action?
* What outcome did you expect instead?
*** End of the template - remove these template lines ***
Niels Thykier
Dec 17, 2022, 6:30:04 AM12/17/22
to
On Fri, 16 Dec 2022 13:13:34 -0500 Garrett Kajmowicz
<garrett....@sophos.com> wrote:
> Package: debhelper
> Version: 13.6ubuntu1
> Severity: wishlist
>
> Dear Maintainer,
>
Hi Garret
<garrett....@sophos.com> wrote:
> Package: debhelper
> Version: 13.6ubuntu1
> Severity: wishlist
>
> Dear Maintainer,
>
I have CC'ed Guillem Jover because *if* this is a change to be
supported, then it would need dpkg and debhelper to support it.
> I'm working on a use-case which can be generalized as wanting to be able to
> perform a build with debhelper in a read-only source directory. Not simply
> that the files themselves are read-only, but that the full directory
>
Ok. As you have discovered, having a read-only source directory is not
a use-case that the Debian build stack supports at the moment. The
build process is largely working with the assumption that the source
package is copied to a writeable directory and then built there (i.e.,
you have a writeable base directory and then unpack the source package
into a subfolder of that directory).
For this to be possible, it would require two "independent" features in
the Debian build stack:
C1) Provide a writeable directory for the output artefacts (the
produced .debs, the .changes file etc.)
- This would enable to the parent directory be read-only.
C2) Provide a writeable directory for the build process itself.
- This would enable the source directory itself to be read-only.
=> There will be packages that *cannot* support C2 as they only
support "in-source" built (which is probably why we have the
current setup in the first place).
The C1) case is something that could be useful for Debian itself. There
are some tools that want to choose the location of the produced
artefacts. It has been a low priority wish for me to have this solved
"eventually".
For the C2) case, it is not that I cannot see possibilities in it. I am
more whether it outweighs the cost of doing it.
=> If we are going for C2, I would expect that you provide CI test
cases to ensure the feature does not regress (at least for debhelper
for the gitlab CI pipeline). Otherwise, it is going to be a game of
"whack-a-mole".
=> For C2, You may also have to do patches and tests for dh-autoreconf
and dh-strip-nondeterminism as they are part of the default dh
pipeline (and not maintained by me nor Guillem).
One possible solution to this is to have:
1) dpkg-buildpackage gets two new parameters. One for the artifact
output directory and one for the writeable scratch directory.
2) dpkg-buildpackage would export these directories as environment
variables for the build process (debian/rules + debhelper).
3) debhelper (and anything in debian/rules) would pick up these
environment variables to write to the correct places.
4) dpkg-buildpackage would pass relevant options to dpkg-genbuildinfo
and dpkg-genchanges (etc.), so they are aware as well.
@Guillem: What is your views on this?
> Many of the required capabilities are already supported. For example, the
> --builddirectory flag allows the building step to be placed in a different
> directory.
they are not aimed at the thing you are doing. Notably the dh_* -P
option is really painful to use at scale (if you have multiple binary
packages in the same source package).
> [...]
>
> The debhelper scripts are causing a few issues. The immediate issue which has
> led me to file this ticket is that debhelper really wants to write log files
> into the source directory. This results in an error such as:
>
> dh_autoreconf_clean: error: failed to write to debian/<package>.debhelper.log: Read-only file system
>
> I tried to work around this by hacking up the debhelper scripts in the Docker
> image by doing:
>
> RUN sed -i -e 's!"debian/${ext}debhelper.log"!"/working/debian/${ext}debhelper.log"!g' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
> RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/share/perl5/Debian/Debhelper/SequencerUtil.pm
> RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/bin/dh_clean
>
> This failed because the script was unable to find the log file to delete, with
> error message:
>
> dh_autoreconf_clean: error: failed to write to /working/debian/<package>.debhelper.log: No such file or directory
>
The code assumes the directory exist. I assume a mkdir -p
> The debhelper scripts are causing a few issues. The immediate issue which has
> led me to file this ticket is that debhelper really wants to write log files
> into the source directory. This results in an error such as:
>
> dh_autoreconf_clean: error: failed to write to debian/<package>.debhelper.log: Read-only file system
>
> I tried to work around this by hacking up the debhelper scripts in the Docker
> image by doing:
>
> RUN sed -i -e 's!"debian/${ext}debhelper.log"!"/working/debian/${ext}debhelper.log"!g' /usr/share/perl5/Debian/Debhelper/Dh_Lib.pm
> RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/share/perl5/Debian/Debhelper/SequencerUtil.pm
> RUN sed -i -e 's!debian/\*.debhelper.log!/working/debian/\*.debhelper.log!g' /usr/bin/dh_clean
>
> This failed because the script was unable to find the log file to delete, with
> error message:
>
> dh_autoreconf_clean: error: failed to write to /working/debian/<package>.debhelper.log: No such file or directory
>
/working/debian would solve that immediate error. However, you would
hit the next wall one step further ahead, so do not expect much progress
from that "mkdir -p".
Also, if you know you have "clean" source directory, you can skip the
clean process with "dpkg-buildpackage -nc".
> [...]
>
> One other possibly-related item of note (I can file a separate bug if you like)
> and which may have an existing solution:
> Passing in --buildinfo-option=-u/out still results in a -O option which
> refers to the wrong directory. That is, I get the following:
> dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
>
> This resulted from an experiment where I was copying the source code to a
> temporary directory with read/write permissions, but where the parent
> directory was not writable.
>
> Thank you for your time,
>
> - Garrett
>
>
> [...]
> One other possibly-related item of note (I can file a separate bug if you like)
> and which may have an existing solution:
> Passing in --buildinfo-option=-u/out still results in a -O option which
> refers to the wrong directory. That is, I get the following:
> dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
>
> This resulted from an experiment where I was copying the source code to a
> temporary directory with read/write permissions, but where the parent
> directory was not writable.
>
> Thank you for your time,
>
> - Garrett
>
>
If it is a bug, then it is a separate one for dpkg. I will let Guillem
comment on that.
Hope that was useful.
Thanks,
~Niels
Niels Thykier
Dec 22, 2022, 4:50:03 AM12/22/22
to
Garrett Kajmowicz:
>> [...]
> by itself would still let me tidy some things up and so would be appreciated.
>
Good to know. I also suspect C1 is easier for us to implement in a short
time frame.
>> For the C2) case, it is not that I cannot see possibilities in it. I am more
>> whether it outweighs the cost of doing it.
>>
>> => If we are going for C2, I would expect that you provide CI test
>> cases to ensure the feature does not regress (at least for debhelper
>> for the gitlab CI pipeline). Otherwise, it is going to be a game of
>> "whack-a-mole".
>
> I have little experience with either gitlab or perl. Still, if you provide me a
> few pointers I can certainly throw a few hours into developing some CI
> tests for this.
>
It does not have to be perl. A shell script that can setup a test
package that is built with dh is sufficient inside gitlab CI is fine.
In case you have experience with GitHub CI, then it is a different
syntax but otherwise the same concept. As an example, the current
debhelper CI snippet for testing changes is:
"""
tests-unstable:
stage: test
image: debian:unstable
script:
- apt-get update
- apt-get build-dep -y .
- dpkg-buildpackage -us -uc -tc
"""
In theory, you would insert snippets in the "script:" part to setup the
source package that debhelper is supposed to build (with the features
you want to test), create the output directory, mark the source
read-only somehow and then cd into followed by dpkg-buildpackage.
This would suffice as a single black box test. Probably we would need a
battery of them, so you would wrap it in a script to try different test
cases but that can be done in shell or whatever you are comfortable.
>> => For C2, You may also have to do patches and tests for dh-autoreconf
>> and dh-strip-nondeterminism as they are part of the default dh
>> pipeline (and not maintained by me nor Guillem).
>
> I can certainly take a stab at it. The required changes for both of these
> appear to be straight-forward: use the new parameter passed from
> above (however architected) as-needed for the paths being processed.
>
> Side note: patching ltmain.sh in-place strikes me as ... worrisome.
>
Actually, come to think of it, The dh-autoreconf tool's purpose is to
*replace* (regenerate) the "configure" script from latest sources. So
this pretty much implies that the source directory have to be writeable
if dh-autoreconf is to do anything.
So as long as it properly no-ops for non-autoconf packages, I think we
should be fine here.
>> One possible solution to this is to have:
>>
>> 1) dpkg-buildpackage gets two new parameters. One for the artifact
>> output directory and one for the writeable scratch directory.
>>
>> 2) dpkg-buildpackage would export these directories as environment
>> variables for the build process (debian/rules + debhelper).
>>
>> 3) debhelper (and anything in debian/rules) would pick up these
>> environment variables to write to the correct places.
>>
>> 4) dpkg-buildpackage would pass relevant options to dpkg-genbuildinfo
>> and dpkg-genchanges (etc.), so they are aware as well.
>
> Should debuild itself be modified to support passing in these build
> Parameters as well?
>
In theory, yes. But really, all "*-buildpackage"-like wrapper tools
should do that.
Though for starters, lets get support in dpkg-buildpackage first and
then worry about patching the other tools.
>> @Guillem: What is your views on this?
>>
>>
> directory unless manually overridden.
>
No, the "builddirectory" is only for the "upstream" build system.
>>> [...]
>
> Okay. I'll await his response as well.
>
>> Hope that was useful.
>
> It's fantastic.
>
> Thank you!
>
> - Garrett
Glad you are excited, though keep in mind that there are no guarantees
at this point. If (for whatever reason), dpkg cannot/will not support
this change, then there is not a lot I can do in debhelper to solve
this. (And also, we are volunteers, so we generally do not offer
guarantees either even if we can/want to support a feature)
Thanks,
~Niels
>> [...]
>>
>> The C1) case is something that could be useful for Debian itself. There are
>> some tools that want to choose the location of the produced artefacts. It has
>> been a low priority wish for me to have this solved "eventually".
>
> I'm using multiple work-arounds for the project I am working on. The C1 work
>> The C1) case is something that could be useful for Debian itself. There are
>> some tools that want to choose the location of the produced artefacts. It has
>> been a low priority wish for me to have this solved "eventually".
>
> by itself would still let me tidy some things up and so would be appreciated.
>
Good to know. I also suspect C1 is easier for us to implement in a short
time frame.
>> For the C2) case, it is not that I cannot see possibilities in it. I am more
>> whether it outweighs the cost of doing it.
>>
>> => If we are going for C2, I would expect that you provide CI test
>> cases to ensure the feature does not regress (at least for debhelper
>> for the gitlab CI pipeline). Otherwise, it is going to be a game of
>> "whack-a-mole".
>
> few pointers I can certainly throw a few hours into developing some CI
> tests for this.
>
It does not have to be perl. A shell script that can setup a test
package that is built with dh is sufficient inside gitlab CI is fine.
In case you have experience with GitHub CI, then it is a different
syntax but otherwise the same concept. As an example, the current
debhelper CI snippet for testing changes is:
"""
tests-unstable:
stage: test
image: debian:unstable
script:
- apt-get update
- apt-get build-dep -y .
- dpkg-buildpackage -us -uc -tc
"""
In theory, you would insert snippets in the "script:" part to setup the
source package that debhelper is supposed to build (with the features
you want to test), create the output directory, mark the source
read-only somehow and then cd into followed by dpkg-buildpackage.
This would suffice as a single black box test. Probably we would need a
battery of them, so you would wrap it in a script to try different test
cases but that can be done in shell or whatever you are comfortable.
>> => For C2, You may also have to do patches and tests for dh-autoreconf
>> and dh-strip-nondeterminism as they are part of the default dh
>> pipeline (and not maintained by me nor Guillem).
>
> appear to be straight-forward: use the new parameter passed from
> above (however architected) as-needed for the paths being processed.
>
> Side note: patching ltmain.sh in-place strikes me as ... worrisome.
>
Actually, come to think of it, The dh-autoreconf tool's purpose is to
*replace* (regenerate) the "configure" script from latest sources. So
this pretty much implies that the source directory have to be writeable
if dh-autoreconf is to do anything.
So as long as it properly no-ops for non-autoconf packages, I think we
should be fine here.
>> One possible solution to this is to have:
>>
>> 1) dpkg-buildpackage gets two new parameters. One for the artifact
>> output directory and one for the writeable scratch directory.
>>
>> 2) dpkg-buildpackage would export these directories as environment
>> variables for the build process (debian/rules + debhelper).
>>
>> 3) debhelper (and anything in debian/rules) would pick up these
>> environment variables to write to the correct places.
>>
>> 4) dpkg-buildpackage would pass relevant options to dpkg-genbuildinfo
>> and dpkg-genchanges (etc.), so they are aware as well.
>
> Parameters as well?
>
In theory, yes. But really, all "*-buildpackage"-like wrapper tools
should do that.
Though for starters, lets get support in dpkg-buildpackage first and
then worry about patching the other tools.
>> @Guillem: What is your views on this?
>>
>>
>>> Many of the required capabilities are already supported. For example,
>>> the --builddirectory flag allows the building step to be placed in a
>>> different directory.
>>
>>> the --builddirectory flag allows the building step to be placed in a
>>> different directory.
>>
>> While there are many options that package can use to tweak locations, they
>> are not aimed at the thing you are doing. Notably the dh_* -P option is really
>> painful to use at scale (if you have multiple binary packages in the same
>> source package).
>
> Huh. I assumed that these would be automatically use builddirectory/debian/<package>
>> are not aimed at the thing you are doing. Notably the dh_* -P option is really
>> painful to use at scale (if you have multiple binary packages in the same
>> source package).
>
> directory unless manually overridden.
>
No, the "builddirectory" is only for the "upstream" build system.
>>> [...]
>
> Okay. I'll await his response as well.
>
>> Hope that was useful.
>
> It's fantastic.
>
> Thank you!
>
> - Garrett
Glad you are excited, though keep in mind that there are no guarantees
at this point. If (for whatever reason), dpkg cannot/will not support
this change, then there is not a lot I can do in debhelper to solve
this. (And also, we are volunteers, so we generally do not offer
guarantees either even if we can/want to support a feature)
Thanks,
~Niels
Guillem Jover
Dec 23, 2022, 7:10:04 AM12/23/22
to
Hi!
On Sat, 2022-12-17 at 12:07:22 +0100, Niels Thykier wrote:
something like this, it seems a bit like one of those gargantuan
projects we might end up with. So, I guess I'd also like to understand
the motivations, and perhaps whether other alternatives are not
fulfilling? :)
I've not checked very deep on what might be blocking this, or whether
there might be other difficulties, so do not take the following as
extensive.
> For this to be possible, it would require two "independent" features in the
> Debian build stack:
>
> C1) Provide a writeable directory for the output artefacts (the
> produced .debs, the .changes file etc.)
> - This would enable to the parent directory be read-only.
This would currently be blocked by at least something like #657401,
I think
> C2) Provide a writeable directory for the build process itself.
> - This would enable the source directory itself to be read-only.
>
> => There will be packages that *cannot* support C2 as they only
> support "in-source" built (which is probably why we have the
> current setup in the first place).
I think this is partially blocked on something like
<https://lists.debian.org/debian-devel/2020/03/msg00085.html>, and
because several of the currently written files are part of the
expected interface. :/
> The C1) case is something that could be useful for Debian itself. There are
> some tools that want to choose the location of the produced artefacts. It
> has been a low priority wish for me to have this solved "eventually".
The overall problem, IMO, is the original packaging design we've got
in front of us, with package building being inside-out, where the build
is mainly driven by the package itself, instead of an external driver.
We have discussed this with Niels in the past, and we have some WIP work
(which I should merge the dpkg side of and email out, something I had
actually in mind talking with Niels already the past weeks!). But I'm not
seeing this (at least right now) being used by Debian for example. :/
> For the C2) case, it is not that I cannot see possibilities in it. I am more
> whether it outweighs the cost of doing it.
>
> => If we are going for C2, I would expect that you provide CI test
> cases to ensure the feature does not regress (at least for debhelper
> for the gitlab CI pipeline). Otherwise, it is going to be a game of
> "whack-a-mole".
>
> => For C2, You may also have to do patches and tests for dh-autoreconf
> and dh-strip-nondeterminism as they are part of the default dh
> pipeline (and not maintained by me nor Guillem).
The main problem I see there, besides the current pathnames as
interfaces, is that I don't think this can be universally supported by
any package, and as such something like dpkg-buildpackage would need
to way to know whether the package supports this mode, for every and
each package (say a field such as the R³ marking), and if it that does
not get tested per package, then it will tend to break. :/
So I think I'm also currently in the "does it outweigh the cost?"
pondering phase too.
> > Many of the required capabilities are already supported. For example,
> > the --builddirectory flag allows the building step to be placed in a
> > different
> > directory.
>
> While there are many options that package can use to tweak locations, they
> are not aimed at the thing you are doing. Notably the dh_* -P option is
> really painful to use at scale (if you have multiple binary packages in the
> same source package).
Also I'd expect many source packaging (say overrides in debian/rules or
even debian/rules not using dh or debhelper), ancillary packaging tools
(say dh extensions) might expect pathnames to be in specific locations.
> > One other possibly-related item of note (I can file a separate bug if you like)
> > and which may have an existing solution:
> > Passing in --buildinfo-option=-u/out still results in a -O option which
> > refers to the wrong directory. That is, I get the following:
> > dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> > dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
> >
> > This resulted from an experiment where I was copying the source code to
> > a temporary directory with read/write permissions, but where the parent
> > directory was not writable.
try to fix it!
In any case, sorry, I think the outlook might not be very bright, but
as I've said at the beginning, I'm happy to still entertain this.
Thanks,
Guillem
On Sat, 2022-12-17 at 12:07:22 +0100, Niels Thykier wrote:
> On Fri, 16 Dec 2022 13:13:34 -0500 Garrett Kajmowicz wrote:
> > Package: debhelper
> > Version: 13.6ubuntu1
> > Severity: wishlist
> > Package: debhelper
> > Version: 13.6ubuntu1
> > Severity: wishlist
> I have CC'ed Guillem Jover because *if* this is a change to be supported,
> then it would need dpkg and debhelper to support it.
>
> > I'm working on a use-case which can be generalized as wanting to be able
> > to perform a build with debhelper in a read-only source directory. Not
> > simply that the files themselves are read-only, but that the full
> > directory
> > tree is read-only. [...]
>
> Ok. As you have discovered, having a read-only source directory is not a
> use-case that the Debian build stack supports at the moment. The build
> process is largely working with the assumption that the source package is
> copied to a writeable directory and then built there (i.e., you have a
> writeable base directory and then unpack the source package into a subfolder
> of that directory).
Right, and while I'm happy to entertain the possibility to support
> then it would need dpkg and debhelper to support it.
>
> > I'm working on a use-case which can be generalized as wanting to be able
> > to perform a build with debhelper in a read-only source directory. Not
> > simply that the files themselves are read-only, but that the full
> > directory
> > tree is read-only. [...]
>
> Ok. As you have discovered, having a read-only source directory is not a
> use-case that the Debian build stack supports at the moment. The build
> process is largely working with the assumption that the source package is
> copied to a writeable directory and then built there (i.e., you have a
> writeable base directory and then unpack the source package into a subfolder
> of that directory).
something like this, it seems a bit like one of those gargantuan
projects we might end up with. So, I guess I'd also like to understand
the motivations, and perhaps whether other alternatives are not
fulfilling? :)
I've not checked very deep on what might be blocking this, or whether
there might be other difficulties, so do not take the following as
extensive.
> For this to be possible, it would require two "independent" features in the
> Debian build stack:
>
> C1) Provide a writeable directory for the output artefacts (the
> produced .debs, the .changes file etc.)
> - This would enable to the parent directory be read-only.
I think
> C2) Provide a writeable directory for the build process itself.
> - This would enable the source directory itself to be read-only.
>
> => There will be packages that *cannot* support C2 as they only
> support "in-source" built (which is probably why we have the
> current setup in the first place).
<https://lists.debian.org/debian-devel/2020/03/msg00085.html>, and
because several of the currently written files are part of the
expected interface. :/
> The C1) case is something that could be useful for Debian itself. There are
> some tools that want to choose the location of the produced artefacts. It
> has been a low priority wish for me to have this solved "eventually".
in front of us, with package building being inside-out, where the build
is mainly driven by the package itself, instead of an external driver.
We have discussed this with Niels in the past, and we have some WIP work
(which I should merge the dpkg side of and email out, something I had
actually in mind talking with Niels already the past weeks!). But I'm not
seeing this (at least right now) being used by Debian for example. :/
> For the C2) case, it is not that I cannot see possibilities in it. I am more
> whether it outweighs the cost of doing it.
>
> => If we are going for C2, I would expect that you provide CI test
> cases to ensure the feature does not regress (at least for debhelper
> for the gitlab CI pipeline). Otherwise, it is going to be a game of
> "whack-a-mole".
>
> => For C2, You may also have to do patches and tests for dh-autoreconf
> and dh-strip-nondeterminism as they are part of the default dh
> pipeline (and not maintained by me nor Guillem).
interfaces, is that I don't think this can be universally supported by
any package, and as such something like dpkg-buildpackage would need
to way to know whether the package supports this mode, for every and
each package (say a field such as the R³ marking), and if it that does
not get tested per package, then it will tend to break. :/
So I think I'm also currently in the "does it outweigh the cost?"
pondering phase too.
> > Many of the required capabilities are already supported. For example,
> > the --builddirectory flag allows the building step to be placed in a
> > different
> > directory.
>
> While there are many options that package can use to tweak locations, they
> are not aimed at the thing you are doing. Notably the dh_* -P option is
> really painful to use at scale (if you have multiple binary packages in the
> same source package).
even debian/rules not using dh or debhelper), ancillary packaging tools
(say dh extensions) might expect pathnames to be in specific locations.
> > One other possibly-related item of note (I can file a separate bug if you like)
> > and which may have an existing solution:
> > Passing in --buildinfo-option=-u/out still results in a -O option which
> > refers to the wrong directory. That is, I get the following:
> > dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> > dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
> >
> > This resulted from an experiment where I was copying the source code to
> > a temporary directory with read/write permissions, but where the parent
> > directory was not writable.
> If it is a bug, then it is a separate one for dpkg. I will let Guillem
> comment on that.
Thanks, I've not yet checked what might be going on, but I'll do and
> comment on that.
try to fix it!
In any case, sorry, I think the outlook might not be very bright, but
as I've said at the beginning, I'm happy to still entertain this.
Thanks,
Guillem
Guillem Jover
Dec 24, 2022, 12:10:03 PM12/24/22
to
On Sat, 2022-12-17 at 12:07:22 +0100, Niels Thykier wrote:
> On Fri, 16 Dec 2022 13:13:34 -0500 Garrett Kajmowicz wrote:
> > One other possibly-related item of note (I can file a separate bug if you like)
> > and which may have an existing solution:
> > Passing in --buildinfo-option=-u/out still results in a -O option which
> > refers to the wrong directory. That is, I get the following:
> > dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> > dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
> >
> > This resulted from an experiment where I was copying the source code to
> > a temporary directory with read/write permissions, but where the parent
> > directory was not writable.
> > One other possibly-related item of note (I can file a separate bug if you like)
> > and which may have an existing solution:
> > Passing in --buildinfo-option=-u/out still results in a -O option which
> > refers to the wrong directory. That is, I get the following:
> > dpkg-genbuildinfo -u/out --build=binary -O../<package>_2.0.6-1_amd64.buildinfo
> > dpkg-genbuildinfo: error: cannot write ../<package>_2.0.6-1_amd64.buildinfo.new: Permission denied
> >
> > This resulted from an experiment where I was copying the source code to
> > a temporary directory with read/write permissions, but where the parent
> > directory was not writable.
> If it is a bug, then it is a separate one for dpkg. I will let Guillem
> comment on that.
After checking this, I think I'd consider this at most a missing
> comment on that.
feature, related to the overall request here, and as part of that to be
able to specify a different global "upload-files-dir".
But for now I think you should be able to overcome this by using the
dpkg-buildpackage --buildinfo-file option.
Thanks,
Guillem
0 new messages