Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
build profile proposal: nogir (second try)
1 view
Skip to first unread message
Simon McVittie
Jan 17, 2024, 5:10:03 PM1/17/24
to
Last year, Helmut Grohne proposed a nogir build profile to help with
cross-compiling the GLib ecosystem:
<https://lists.debian.org/debian-devel/2023/04/msg00056.html>.
After some discussion on #1030223, I have a revised proposal, with the
same name but slightly different rules:
profile name: nogir
content changes: Y/N (ideally N)
package set changes: Y
description: Disable GObject-Introspection bindings (.gir, .typelib).
See the GObject-Introspection mini-policy for details.
For background, GObject-Introspection provides machine-readable
information about C/C++ code that follows GLib's conventions. There are
two formats: GIR XML is a text format with full information for compiled
bindings (C++, Rust, Haskell), and typelibs are a more efficient binary
format with a subset of that information for dynamic bindings (Python,
JavaScript, Perl). There is a compiler to do the lossy conversion from
GIR XML into typelibs, but creating the GIR XML is the part that has
dependencies we sometimes want to avoid, so disabling the typelibs but
leaving the GIR XML in place is not useful.
Public typelibs (Foo-1.typelib) are already shipped in their own package,
gir1.2-foo-1, so they are easy to disable with a build profile.
Public GIR XML (Foo-1.gir) is normally in the -dev package alongside the
C headers, but recent versions of gobject-introspection define a canonical
virtual package name gir1.2-foo-1-dev which can either be a Provides on
the -dev package or an independent binary package.
Here is the draft text that I added to the GObject-Introspection
mini-policy in 1.78.1-11:
----
Source packages that build GIR XML and typelibs may implement the nogir
build profile. This will usually be desirable for libraries that can be
used directly from C/C++ code without needing to use GObject-Introspection.
It will usually not be useful to implement the nogir build profile in
packages where GObject-Introspection is a functional requirement for using
the package, such as gnome-shell.
Ideally, the nogir build profile should disable the build of gir1.2-* and
gir1.2-*-dev packages, and disable all build-dependencies on the
gobject-introspection toolchain and gir1.2-*-dev packages, without
affecting the contents of any other binary package, in particular lib*-dev.
When implemented like this, nogir is a "safe" or "reproducible" build-profile.
However, for many pre-existing source packages, separating GIR XML
from the lib*-dev package into a new gir1.2-*-dev package would be an
incompatible change that needs to be coordinated with dependent packages,
because previously-correct dependent packages would begin to fail to build
from source until they add gir1.2-*-dev to their Build-Depends (even if
the package was built without the nogir profile and the dependency is
a native build). It would also require manual processing by the Debian
archive administrators, to approve the new binary package name.
To mitigate those factors, if the GIR XML is included in a lib*-dev
binary package, the nogir build profile may remove the GIR XML from
that package. The resulting binary package must drop any Provides
matching gir1.2-*-dev that it would have had when built normally. This
will often be automatic: if dh_girepository is used, ${gir:Depends}
and ${gir:Provides} will automatically be emptied for packages that do
not contain any GIR XML or typelibs.
If this is done, dependent packages will continue to build successfully
against any dependency that was not built with nogir, but will require
changes (namely adding Build-Depends: gir1.2-*-dev <!nogir>) in order to
be sucessfully buildable with dependencies that were built with nogir.
----
I did wonder whether this should be two separate build profiles, but
that seems like it's probably unnecessary complexity.
smcv
cross-compiling the GLib ecosystem:
<https://lists.debian.org/debian-devel/2023/04/msg00056.html>.
After some discussion on #1030223, I have a revised proposal, with the
same name but slightly different rules:
profile name: nogir
content changes: Y/N (ideally N)
package set changes: Y
description: Disable GObject-Introspection bindings (.gir, .typelib).
See the GObject-Introspection mini-policy for details.
For background, GObject-Introspection provides machine-readable
information about C/C++ code that follows GLib's conventions. There are
two formats: GIR XML is a text format with full information for compiled
bindings (C++, Rust, Haskell), and typelibs are a more efficient binary
format with a subset of that information for dynamic bindings (Python,
JavaScript, Perl). There is a compiler to do the lossy conversion from
GIR XML into typelibs, but creating the GIR XML is the part that has
dependencies we sometimes want to avoid, so disabling the typelibs but
leaving the GIR XML in place is not useful.
Public typelibs (Foo-1.typelib) are already shipped in their own package,
gir1.2-foo-1, so they are easy to disable with a build profile.
Public GIR XML (Foo-1.gir) is normally in the -dev package alongside the
C headers, but recent versions of gobject-introspection define a canonical
virtual package name gir1.2-foo-1-dev which can either be a Provides on
the -dev package or an independent binary package.
Here is the draft text that I added to the GObject-Introspection
mini-policy in 1.78.1-11:
----
Source packages that build GIR XML and typelibs may implement the nogir
build profile. This will usually be desirable for libraries that can be
used directly from C/C++ code without needing to use GObject-Introspection.
It will usually not be useful to implement the nogir build profile in
packages where GObject-Introspection is a functional requirement for using
the package, such as gnome-shell.
Ideally, the nogir build profile should disable the build of gir1.2-* and
gir1.2-*-dev packages, and disable all build-dependencies on the
gobject-introspection toolchain and gir1.2-*-dev packages, without
affecting the contents of any other binary package, in particular lib*-dev.
When implemented like this, nogir is a "safe" or "reproducible" build-profile.
However, for many pre-existing source packages, separating GIR XML
from the lib*-dev package into a new gir1.2-*-dev package would be an
incompatible change that needs to be coordinated with dependent packages,
because previously-correct dependent packages would begin to fail to build
from source until they add gir1.2-*-dev to their Build-Depends (even if
the package was built without the nogir profile and the dependency is
a native build). It would also require manual processing by the Debian
archive administrators, to approve the new binary package name.
To mitigate those factors, if the GIR XML is included in a lib*-dev
binary package, the nogir build profile may remove the GIR XML from
that package. The resulting binary package must drop any Provides
matching gir1.2-*-dev that it would have had when built normally. This
will often be automatic: if dh_girepository is used, ${gir:Depends}
and ${gir:Provides} will automatically be emptied for packages that do
not contain any GIR XML or typelibs.
If this is done, dependent packages will continue to build successfully
against any dependency that was not built with nogir, but will require
changes (namely adding Build-Depends: gir1.2-*-dev <!nogir>) in order to
be sucessfully buildable with dependencies that were built with nogir.
----
I did wonder whether this should be two separate build profiles, but
that seems like it's probably unnecessary complexity.
smcv
Simon McVittie
Jan 17, 2024, 6:40:04 PM1/17/24
to
On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
> Am 17.01.24 um 23:00 schrieb Simon McVittie:
> source packages into a separate gir1.2-foo-dev (in the long run) ?
That's a good question, and I don't have an easy answer for it. The
tradeoff is:
- having the GIR XML in libfoo-dev means fewer binary packages and
therefore smaller Packages metadata;
- having a separate (non-virtual) gir1.2-foo-1-dev means we can "cleanly"
turn off GIR/typelibs in cases when they're not needed, and means
libfoo-dev is a bit smaller and with fewer dependencies
It's analogous to the choice between one big -dev package (libcairo2-dev,
libwayland-dev) or multiple smaller -dev packages (libportal*-dev) for a
source package with more than one shared library.
The larger, more widely-used and lower-level the library is, the more I
would be inclined to opt for the approach with extra binary packages
- for example splitting out gir1.2-gtk-4.0-dev from libgtk-4-dev
seems more desirable than splitting out gir1.2-shumate-1.0-dev from
libshumate-dev. Separating out the GIR XML is more interesting for
packages that are involved in bootstrapping, or for packages that someone
will frequently want to cross-compile, particularly the ones you'll want
to cross-compile early in the development of a new port when tools like
qemu-user might not be able to target it.
In the case where the GIR XML is in libfoo-dev, asking for it to have
Provides: gir1.2-foo-1-dev means that dependent packages can depend on the
systematic gir1.2-foo-1-dev name, and then will work correctly either way.
The only package where I'm sure that I intend to separate out the GIR
XML in the short term is src:glib2.0, where for historical reasons
gir1.2-glib-2.0 has been built by src:gobject-introspection until
now. I'm most of the way through preparing a version of glib2.0
2.79.x for experimental that takes over gir1.2-glib-2.0{,-dev}
from src:gobject-introspection, and I definitely don't want to fold
gir1.2-glib-2.0-dev into libglib2.0-dev, because GLib's position at the
bottom of the GNOME stack makes it particularly important that we can
still bootstrap and cross-compile it.
smcv
> Am 17.01.24 um 23:00 schrieb Simon McVittie:
> > Public GIR XML (Foo-1.gir) is normally in the -dev package alongside the
> > C headers, but recent versions of gobject-introspection define a canonical
> > virtual package name gir1.2-foo-1-dev which can either be a Provides on
> > the -dev package or an independent binary package.
>
> Does this mean we should should split out the .gir XML files from existing
> > C headers, but recent versions of gobject-introspection define a canonical
> > virtual package name gir1.2-foo-1-dev which can either be a Provides on
> > the -dev package or an independent binary package.
>
> source packages into a separate gir1.2-foo-dev (in the long run) ?
That's a good question, and I don't have an easy answer for it. The
tradeoff is:
- having the GIR XML in libfoo-dev means fewer binary packages and
therefore smaller Packages metadata;
- having a separate (non-virtual) gir1.2-foo-1-dev means we can "cleanly"
turn off GIR/typelibs in cases when they're not needed, and means
libfoo-dev is a bit smaller and with fewer dependencies
It's analogous to the choice between one big -dev package (libcairo2-dev,
libwayland-dev) or multiple smaller -dev packages (libportal*-dev) for a
source package with more than one shared library.
The larger, more widely-used and lower-level the library is, the more I
would be inclined to opt for the approach with extra binary packages
- for example splitting out gir1.2-gtk-4.0-dev from libgtk-4-dev
seems more desirable than splitting out gir1.2-shumate-1.0-dev from
libshumate-dev. Separating out the GIR XML is more interesting for
packages that are involved in bootstrapping, or for packages that someone
will frequently want to cross-compile, particularly the ones you'll want
to cross-compile early in the development of a new port when tools like
qemu-user might not be able to target it.
In the case where the GIR XML is in libfoo-dev, asking for it to have
Provides: gir1.2-foo-1-dev means that dependent packages can depend on the
systematic gir1.2-foo-1-dev name, and then will work correctly either way.
The only package where I'm sure that I intend to separate out the GIR
XML in the short term is src:glib2.0, where for historical reasons
gir1.2-glib-2.0 has been built by src:gobject-introspection until
now. I'm most of the way through preparing a version of glib2.0
2.79.x for experimental that takes over gir1.2-glib-2.0{,-dev}
from src:gobject-introspection, and I definitely don't want to fold
gir1.2-glib-2.0-dev into libglib2.0-dev, because GLib's position at the
bottom of the GNOME stack makes it particularly important that we can
still bootstrap and cross-compile it.
smcv
Johannes Schauer Marin Rodrigues
Jan 18, 2024, 4:50:03 AM1/18/24
to
Hi,
On 2024-01-18 00:38, Simon McVittie wrote:
> On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
>> Am 17.01.24 um 23:00 schrieb Simon McVittie:
>> > Public GIR XML (Foo-1.gir) is normally in the -dev package alongside the
>> > C headers, but recent versions of gobject-introspection define a canonical
>> > virtual package name gir1.2-foo-1-dev which can either be a Provides on
>> > the -dev package or an independent binary package.
>>
>> Does this mean we should should split out the .gir XML files from
>> existing
>> source packages into a separate gir1.2-foo-dev (in the long run) ?
>
> That's a good question, and I don't have an easy answer for it. The
> tradeoff is:
>
> - having the GIR XML in libfoo-dev means fewer binary packages and
> therefore smaller Packages metadata;
>
> - having a separate (non-virtual) gir1.2-foo-1-dev means we can
> "cleanly"
> turn off GIR/typelibs in cases when they're not needed, and means
> libfoo-dev is a bit smaller and with fewer dependencies
maybe it makes sense to get back to Helmut's original [message] that
Simon linked to a few mails ago:
> Why?
> gobject-introspection is one of the few and high popcon components that
> poses resistant to cross compilation. gir packages are often added to
> existing source packages for e.g. libraries and their presence makes
> cross building fail. Often enough, these gir packages are not needed
> for
> a particular application, so skipping them would allow cross building
> the library. The profile also allows fixing cross build problems in
> packages shipping typelib files such that when we get a solution for
> typelib generation, those packages will not have other bugs.
[message] https://lists.debian.org/debian-devel/2023/04/msg00056.html
When deciding for trade-offs we should probably go back and look at the
"Why" for this whole effort. Back when Helmut wrote this message, Simon
hadn't yet implemented the mechanism that would use qemu to allow for
cross-compilation to supported architectures. For when it works, it
works *really* well. Before Simon's efforts, I had to build packages
using gobject-introspection "natively" using qemu user-mode emulation to
get foreign architecture binaries. In my situation this took 1.5 hours.
With what is currently in Debian unstable, I am able to compile the same
packages in under 4 minutes. This is a 20-fold speed-up (Thank you Simon
and Helmut!! <3). But of course there are situations where the qemu
method is not applicable. Maybe nogir build profile and the package
splitting should be limited to carefully selected situations where it is
deemed necessary by bootstrappers or cross-builders that doing so has
some real-world benefits? I don't think it makes much sense to add nogir
profiles and package splits where they do not serve much of a purpose.
Doing so would mean to just get the cost of doing so without any
benefit.
Thanks!
cheers, josch
On 2024-01-18 00:38, Simon McVittie wrote:
> On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
>> Am 17.01.24 um 23:00 schrieb Simon McVittie:
>> > Public GIR XML (Foo-1.gir) is normally in the -dev package alongside the
>> > C headers, but recent versions of gobject-introspection define a canonical
>> > virtual package name gir1.2-foo-1-dev which can either be a Provides on
>> > the -dev package or an independent binary package.
>>
>> Does this mean we should should split out the .gir XML files from
>> existing
>> source packages into a separate gir1.2-foo-dev (in the long run) ?
>
> That's a good question, and I don't have an easy answer for it. The
> tradeoff is:
>
> - having the GIR XML in libfoo-dev means fewer binary packages and
> therefore smaller Packages metadata;
>
> - having a separate (non-virtual) gir1.2-foo-1-dev means we can
> "cleanly"
> turn off GIR/typelibs in cases when they're not needed, and means
> libfoo-dev is a bit smaller and with fewer dependencies
Simon linked to a few mails ago:
> Why?
> gobject-introspection is one of the few and high popcon components that
> poses resistant to cross compilation. gir packages are often added to
> existing source packages for e.g. libraries and their presence makes
> cross building fail. Often enough, these gir packages are not needed
> for
> a particular application, so skipping them would allow cross building
> the library. The profile also allows fixing cross build problems in
> packages shipping typelib files such that when we get a solution for
> typelib generation, those packages will not have other bugs.
[message] https://lists.debian.org/debian-devel/2023/04/msg00056.html
When deciding for trade-offs we should probably go back and look at the
"Why" for this whole effort. Back when Helmut wrote this message, Simon
hadn't yet implemented the mechanism that would use qemu to allow for
cross-compilation to supported architectures. For when it works, it
works *really* well. Before Simon's efforts, I had to build packages
using gobject-introspection "natively" using qemu user-mode emulation to
get foreign architecture binaries. In my situation this took 1.5 hours.
With what is currently in Debian unstable, I am able to compile the same
packages in under 4 minutes. This is a 20-fold speed-up (Thank you Simon
and Helmut!! <3). But of course there are situations where the qemu
method is not applicable. Maybe nogir build profile and the package
splitting should be limited to carefully selected situations where it is
deemed necessary by bootstrappers or cross-builders that doing so has
some real-world benefits? I don't think it makes much sense to add nogir
profiles and package splits where they do not serve much of a purpose.
Doing so would mean to just get the cost of doing so without any
benefit.
Thanks!
cheers, josch
Helmut Grohne
Jan 21, 2024, 9:30:05 AM1/21/24
to
On Wed, Jan 17, 2024 at 11:38:09PM +0000, Simon McVittie wrote:
> On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
> > Does this mean we should should split out the .gir XML files from existing
> > source packages into a separate gir1.2-foo-dev (in the long run) ?
>
> That's a good question, and I don't have an easy answer for it. The
> tradeoff is:
>
> - having the GIR XML in libfoo-dev means fewer binary packages and
> therefore smaller Packages metadata;
>
> - having a separate (non-virtual) gir1.2-foo-1-dev means we can "cleanly"
> turn off GIR/typelibs in cases when they're not needed, and means
> libfoo-dev is a bit smaller and with fewer dependencies
Really, I think the main advantage of splitting them out into real
> On Wed, 17 Jan 2024 at 23:15:03 +0100, Matthias Geiger wrote:
> > Does this mean we should should split out the .gir XML files from existing
> > source packages into a separate gir1.2-foo-dev (in the long run) ?
>
> That's a good question, and I don't have an easy answer for it. The
> tradeoff is:
>
> - having the GIR XML in libfoo-dev means fewer binary packages and
> therefore smaller Packages metadata;
>
> - having a separate (non-virtual) gir1.2-foo-1-dev means we can "cleanly"
> turn off GIR/typelibs in cases when they're not needed, and means
> libfoo-dev is a bit smaller and with fewer dependencies
packages is the additional QA that we get. With the Provides-mechanism,
consumers will often miss the additional gir1.2-*-dev build dependency
that is required and adding those back will be a permanent duty of cross
build porters.
> It's analogous to the choice between one big -dev package (libcairo2-dev,
> libwayland-dev) or multiple smaller -dev packages (libportal*-dev) for a
> source package with more than one shared library.
> The larger, more widely-used and lower-level the library is, the more I
> would be inclined to opt for the approach with extra binary packages
> - for example splitting out gir1.2-gtk-4.0-dev from libgtk-4-dev
> seems more desirable than splitting out gir1.2-shumate-1.0-dev from
> libshumate-dev. Separating out the GIR XML is more interesting for
> packages that are involved in bootstrapping, or for packages that someone
> will frequently want to cross-compile, particularly the ones you'll want
> to cross-compile early in the development of a new port when tools like
> qemu-user might not be able to target it.
concur with that. The provides mechanism seems easier for maintainers
and so I'd recommend doing that, then changing to the split mechanism
where we deem it useful.
> In the case where the GIR XML is in libfoo-dev, asking for it to have
> Provides: gir1.2-foo-1-dev means that dependent packages can depend on the
> systematic gir1.2-foo-1-dev name, and then will work correctly either way.
correctly depend on these virtual facilities. I fear the simplest way is
actually splitting the binary packages. Does anyone have a better idea?
> The only package where I'm sure that I intend to separate out the GIR
> XML in the short term is src:glib2.0, where for historical reasons
> gir1.2-glib-2.0 has been built by src:gobject-introspection until
> now. I'm most of the way through preparing a version of glib2.0
> 2.79.x for experimental that takes over gir1.2-glib-2.0{,-dev}
> from src:gobject-introspection, and I definitely don't want to fold
> gir1.2-glib-2.0-dev into libglib2.0-dev, because GLib's position at the
> bottom of the GNOME stack makes it particularly important that we can
> still bootstrap and cross-compile it.
different source package? glib2.0 is involved with bootstrap at this
time and that works fully automatically *because* it is not involved
with gir. When you add gir, builders have to add the nogir profile (and
thus manually order glib2.0). If you were to split this into two
distinct source packages, you'd remove the need for applying a build
profile and thus automatic bootstrapping continues to work. Of course, I
cannot tell how that impacts the implementation, but given that it
formerly was part of src:gobject-introspection, it cannot be unworkable.
Quite definitely, such a split is not a requirement though.
Helmut
Simon McVittie
Jan 21, 2024, 10:30:03 AM1/21/24
to
On Thu, 18 Jan 2024 at 11:08:30 +0100, Helmut Grohne wrote:
> On Wed, Jan 17, 2024 at 11:38:09PM +0000, Simon McVittie wrote:
> > The only package where I'm sure that I intend to separate out the GIR
> > XML in the short term is src:glib2.0
>
> On Wed, Jan 17, 2024 at 11:38:09PM +0000, Simon McVittie wrote:
> > The only package where I'm sure that I intend to separate out the GIR
> > XML in the short term is src:glib2.0
>
> How annoying would it actually be to split this to a
> different source package?
Really quite annoying. The reason I'm integrating the GIR build into
> different source package?
the glib2.0 source package now is that upstream have done the same,
which allows the different parts to become increasingly tightly-coupled
in future (and I don't think upstream would be putting in this work if
they didn't intend to make use of that ability).
A snapshot of glib2.0 that takes over the GIR build is waiting in NEW
for ftp team approval:
https://ftp-master.debian.org/new/glib2.0_2.79.0+git20240119~62ee8bf6-1.html
https://salsa.debian.org/gnome-team/glib/-/tree/debian/2.79.0+git20240119_62ee8bf6-1?ref_type=tags
It's an upstream git snapshot rather than a formal prerelease, because
quite a lot is still changing in this area, and packaging a snapshot
seemed more applicable to future versions than backporting selected fixes
into the 2.79.0 prerelease; but this is going to be stable API/ABI in
GLib 2.80, due for release in March.
To make GLib properly robust I think we will want the ability to use
lockstep version dependencies here, but uploading two source packages
(with identical source code and different packaging) every time there
is a glib2.0 bug fix, and breaking the release team's ability to binNMU
them independently, seems like a high price to pay for making bootstrap
a little more automatic.
If porters are interested in making bootstrap automatic despite cycles
like this one, I think a better route would be to be able to have
a list of suggested bootstrap steps and build-order considerations,
either centralized in some sort of cross infrastructure or distributed
among packages. I'd be fine with adding something like this to glib2.0,
for example, if it helped:
Bootstrap-Before: dbus, gobject-introspection
Bootstrap-Build-Profiles: nogir, nocheck, noinsttest
Or, if we separated the nogir build profile that I'm proposing here into
two, something like this:
nogir-changing-content
can change content: Y ("unreproducible"/"unsafe" profile)
can change package set: Y
nogir
can change content: N ("reproducible"/"safe" profile)
can change package set: Y
would that allow automatic bootstrapping infrastructure to figure out
that it was both safe and desirable to build glib2.0 with nogir?
(I infer that there must be some sort of infrastructure that knows that
it's safe to build packages with "nocheck,noinsttest", otherwise glib2.0
and dbus are already in a cyclic dependency for their test suites.)
> given that it
> formerly was part of src:gobject-introspection, it cannot be unworkable
to more usual practices, and it worked by copy/pasting and committing
all the doc-comments from glib2.0 into a large "source" file in
gobject-introspection. I don't think anyone really wanted this, but it
was considered a necessary evil to break cyclic dependencies.
As a result of that workaround, project members with a particularly
uncompromising definition of preferred forms for modification have
already required us to duplicate the rest of the src:glib2.0 source into
src:gobject-introspection (implemented as a secondary orig.tar.xz), and
then duplicate all of its copyright information in gobject-introspection's
d/copyright, which was rather unwieldy to say the least.
I personally think a compilation of doc-comments in editable text
form is an entirely valid form of source from which Debian users can
exercise all of their Free Software rights, but it seemed that there
was no consensus on this. Doing a bunch of tedious administrative work
and increasing the size of the gobject-introspection source package by
500% seemed more likely to succeed than getting into a fight about the
semantics of the DFSG with preferred-form-for-modification maximalists,
so I did what I had to do.
I do have limited patience for doing extra volunteer work that I think
is neither fun nor constructive, though, so I would prefer to remove the
situation that made this necessary by making use of the tools that we
have (including build profiles and cross-compiling). I'm sorry if that's
causing extra work for your use-case.
smcv
Helmut Grohne
Jan 21, 2024, 5:40:03 PM1/21/24
to
Hi Simon,
On Sun, Jan 21, 2024 at 03:24:25PM +0000, Simon McVittie wrote:
> > How annoying would it actually be to split this to a
> > different source package?
>
> Really quite annoying. [...]
You gave more than sufficient reason. I won't argue.
> If porters are interested in making bootstrap automatic despite cycles
> like this one, I think a better route would be to be able to have
> a list of suggested bootstrap steps and build-order considerations,
> either centralized in some sort of cross infrastructure or distributed
> among packages. I'd be fine with adding something like this to glib2.0,
> for example, if it helped:
>
> Bootstrap-Before: dbus, gobject-introspection
> Bootstrap-Build-Profiles: nogir, nocheck, noinsttest
We effectively tried the approach of encoding bootstrap-info into
individual packages with stage profiles and that was a bad idea. What
stages are needed can (and does) change. For instance, we no longer need
glibc's stage1 profile and go to stage2 directly. Hence, we try to more
and more use profiles that change a particular aspect of a package in an
obvious and isolated way and externally maintain how these are to be
combined into a successful bootstrap.
> Or, if we separated the nogir build profile that I'm proposing here into
> two, something like this:
>
> nogir-changing-content
> can change content: Y ("unreproducible"/"unsafe" profile)
> can change package set: Y
> nogir
> can change content: N ("reproducible"/"safe" profile)
> can change package set: Y
>
> would that allow automatic bootstrapping infrastructure to figure out
> that it was both safe and desirable to build glib2.0 with nogir?
I've considered this option for other profiles already and did not find
it appealing. Often times, you are interested in enabling the profile
without caring about whether it changes package contents, but such a
split would require you to figure out which of the profiles you need (or
simply both?).
More and more I think that merely documenting which instances of these
profiles are reproducible would be a better approach. I've had this
float as a vague idea since a while:
XS-Reproducible-Profiles: nogir
It's a promise that a source package can issue about a subset of the
profiles it supports. It bears some similarity to "Multi-Arch: foreign"
in the sense that both are promises on how the interface behaves. In
particular, such a declaration would be machine-checkable. We could
simply run an autobuilder that verifies whether such declarations are
practically correct (on amd64).
Bootstrappers do not really need that separation into two different
profile names that you propose. Having the information of which profiles
are reproducible in which source packages (and which packages get
disabled when enabling the profile), is what is needed.
So this is what I prefer, but it still comes at a cost. We're up for
changing lots of packages to declare these headers. And we're up for
setting QA to verify these. I fear I cannot provide the capacity to do
all of this and hence I have not pushed this forward.
Manually ordering glib2.0 in the bootstrap tooling may be annoying, but
that's about it. It still is way less work than any of the alternatives.
> (I infer that there must be some sort of infrastructure that knows that
> it's safe to build packages with "nocheck,noinsttest", otherwise glib2.0
> and dbus are already in a cyclic dependency for their test suites.)
Not really. nocheck and noinsttest are issued by default and simply
assumed to do the right thing in all cases.
> [...] I'm sorry if that's
low compared to the extra work on your side for the alternative. That
makes the choice rather obvious to me. Also having this advance warning
further lowers the cost on my side. You answered my question in way more
detail than expected. Thank you.
Helmut
On Sun, Jan 21, 2024 at 03:24:25PM +0000, Simon McVittie wrote:
> > How annoying would it actually be to split this to a
> > different source package?
>
You gave more than sufficient reason. I won't argue.
> If porters are interested in making bootstrap automatic despite cycles
> like this one, I think a better route would be to be able to have
> a list of suggested bootstrap steps and build-order considerations,
> either centralized in some sort of cross infrastructure or distributed
> among packages. I'd be fine with adding something like this to glib2.0,
> for example, if it helped:
>
> Bootstrap-Before: dbus, gobject-introspection
> Bootstrap-Build-Profiles: nogir, nocheck, noinsttest
individual packages with stage profiles and that was a bad idea. What
stages are needed can (and does) change. For instance, we no longer need
glibc's stage1 profile and go to stage2 directly. Hence, we try to more
and more use profiles that change a particular aspect of a package in an
obvious and isolated way and externally maintain how these are to be
combined into a successful bootstrap.
> Or, if we separated the nogir build profile that I'm proposing here into
> two, something like this:
>
> nogir-changing-content
> can change content: Y ("unreproducible"/"unsafe" profile)
> can change package set: Y
> nogir
> can change content: N ("reproducible"/"safe" profile)
> can change package set: Y
>
> would that allow automatic bootstrapping infrastructure to figure out
> that it was both safe and desirable to build glib2.0 with nogir?
it appealing. Often times, you are interested in enabling the profile
without caring about whether it changes package contents, but such a
split would require you to figure out which of the profiles you need (or
simply both?).
More and more I think that merely documenting which instances of these
profiles are reproducible would be a better approach. I've had this
float as a vague idea since a while:
XS-Reproducible-Profiles: nogir
It's a promise that a source package can issue about a subset of the
profiles it supports. It bears some similarity to "Multi-Arch: foreign"
in the sense that both are promises on how the interface behaves. In
particular, such a declaration would be machine-checkable. We could
simply run an autobuilder that verifies whether such declarations are
practically correct (on amd64).
Bootstrappers do not really need that separation into two different
profile names that you propose. Having the information of which profiles
are reproducible in which source packages (and which packages get
disabled when enabling the profile), is what is needed.
So this is what I prefer, but it still comes at a cost. We're up for
changing lots of packages to declare these headers. And we're up for
setting QA to verify these. I fear I cannot provide the capacity to do
all of this and hence I have not pushed this forward.
Manually ordering glib2.0 in the bootstrap tooling may be annoying, but
that's about it. It still is way less work than any of the alternatives.
> (I infer that there must be some sort of infrastructure that knows that
> it's safe to build packages with "nocheck,noinsttest", otherwise glib2.0
> and dbus are already in a cyclic dependency for their test suites.)
assumed to do the right thing in all cases.
> [...] I'm sorry if that's
> causing extra work for your use-case.
Yes, that's causing extra work on my side, but that extra work is really
low compared to the extra work on your side for the alternative. That
makes the choice rather obvious to me. Also having this advance warning
further lowers the cost on my side. You answered my question in way more
detail than expected. Thank you.
Helmut
Alberto Garcia
Jan 24, 2024, 2:10:04 PM1/24/24
to
On Wed, Jan 17, 2024 at 10:00:35PM +0000, Simon McVittie wrote:
> Here is the draft text that I added to the GObject-Introspection
> mini-policy in 1.78.1-11:
Hi, thanks for the explanation.
> Here is the draft text that I added to the GObject-Introspection
> mini-policy in 1.78.1-11:
A couple of questions about this:
- Are packages that ship gobject-introspection files supposed to have
<!nogir> in the relevant build dependencies (gir1.2-*-dev,
gobject-introspection ?), or is the build profile handling this
automatically?
- Packages using dh_install may have a line with usr/share/gir-1.0 in
their debian/libfoo-dev.install. This will fail if the .gir files
are not generated. What's the recommended way to handle this?
Regards,
Berto
Simon McVittie
Jan 24, 2024, 3:00:03 PM1/24/24
to
As Johannes mentioned earlier in this thread, the first piece of practical
advice on nogir should be: if you don't know that you need to use it,
then perhaps you shouldn't. It's primarily aimed at breaking cycles,
and enabling buildability in lower-level packages during bootstrapping.
(Having said that, at some point I want to convert some not-too-essential
library to this setup, just as a demonstration of how it works in
practice.)
file:///usr/share/doc/gobject-introspection/README.Debian.gz in recent
versions of gobject-introspection has some practical checklists for
making packages with GIR XML and typelibs cross-buildable, and for making
it possible to add the nogir profile. I would say that if you only do
one of those two things to a library, it should be making the library
cross-buildable: that's something that most libraries can benefit from.
On Wed, 24 Jan 2024 at 18:30:02 +0000, Alberto Garcia wrote:
> - Are packages that ship gobject-introspection files supposed to have
> <!nogir> in the relevant build dependencies (gir1.2-*-dev,
> gobject-introspection ?), or is the build profile handling this
> automatically?
If you want to skip a particular build-dependency under this build
profile, you must annotate it with <!nogir>. There is no other way to
skip build-dependencies, and this is true for all build profiles, not
just this one.
The goal of this build profile is more like the other way around,
really. Don't think of it as "packages that ship gobject-introspection
files should implement nogir". Think of it as more like: "packages
that build-depend on gobject-introspection stuff *might* want it to be
possible to turn off that build-dependency: and if they do, the way to
do it without causing FTBFS or breaking dependent packages is to
implement nogir".
For GIR XML and typelibs, I expect that the dependencies that will usually
be marked in this way are gir1.2-*-dev, gobject-introspection, and
libgirepository1.0-dev.
Note that if your package uses gi-docgen to generate documentation,
then that cannot work without enabling introspection, so it might need
something more like:
Build-Depends:
...,
gir1.2-glib-2.0-dev <!nogir> <!nodoc>,
gobject-introspection <!nogir> <!nodoc>,
...,
or even
Build-Depends-Arch:
...,
gir1.2-glib-2.0-dev <!nogir>,
gobject-introspection <!nogir>,
...,
Build-Depends-Indep:
...,
gi-docgen <!nodoc>,
gir1.2-glib-2.0-dev <!nodoc>,
gobject-introspection <!nodoc>,
...,
> - Packages using dh_install may have a line with usr/share/gir-1.0 in
> their debian/libfoo-dev.install. This will fail if the .gir files
> are not generated. What's the recommended way to handle this?
If your GIR XML has been split into a separate gir1.2-foo-0-dev binary
package (see other discussions in this thread), then this isn't a concern
because you can turn off that entire binary package with
Build-Profiles: <!nogir>.
Or, if you're using this build profile in the mode where it changes the
contents of the libfoo-dev binary package (which is non-ideal, but avoids
needing to go through NEW), then there isn't really a recommended way,
but dh-exec is probably the least-bad way.
smcv
advice on nogir should be: if you don't know that you need to use it,
then perhaps you shouldn't. It's primarily aimed at breaking cycles,
and enabling buildability in lower-level packages during bootstrapping.
(Having said that, at some point I want to convert some not-too-essential
library to this setup, just as a demonstration of how it works in
practice.)
file:///usr/share/doc/gobject-introspection/README.Debian.gz in recent
versions of gobject-introspection has some practical checklists for
making packages with GIR XML and typelibs cross-buildable, and for making
it possible to add the nogir profile. I would say that if you only do
one of those two things to a library, it should be making the library
cross-buildable: that's something that most libraries can benefit from.
On Wed, 24 Jan 2024 at 18:30:02 +0000, Alberto Garcia wrote:
> - Are packages that ship gobject-introspection files supposed to have
> <!nogir> in the relevant build dependencies (gir1.2-*-dev,
> gobject-introspection ?), or is the build profile handling this
> automatically?
profile, you must annotate it with <!nogir>. There is no other way to
skip build-dependencies, and this is true for all build profiles, not
just this one.
The goal of this build profile is more like the other way around,
really. Don't think of it as "packages that ship gobject-introspection
files should implement nogir". Think of it as more like: "packages
that build-depend on gobject-introspection stuff *might* want it to be
possible to turn off that build-dependency: and if they do, the way to
do it without causing FTBFS or breaking dependent packages is to
implement nogir".
For GIR XML and typelibs, I expect that the dependencies that will usually
be marked in this way are gir1.2-*-dev, gobject-introspection, and
libgirepository1.0-dev.
Note that if your package uses gi-docgen to generate documentation,
then that cannot work without enabling introspection, so it might need
something more like:
Build-Depends:
...,
gir1.2-glib-2.0-dev <!nogir> <!nodoc>,
gobject-introspection <!nogir> <!nodoc>,
...,
or even
Build-Depends-Arch:
...,
gir1.2-glib-2.0-dev <!nogir>,
gobject-introspection <!nogir>,
...,
Build-Depends-Indep:
...,
gi-docgen <!nodoc>,
gir1.2-glib-2.0-dev <!nodoc>,
gobject-introspection <!nodoc>,
...,
> - Packages using dh_install may have a line with usr/share/gir-1.0 in
> their debian/libfoo-dev.install. This will fail if the .gir files
> are not generated. What's the recommended way to handle this?
package (see other discussions in this thread), then this isn't a concern
because you can turn off that entire binary package with
Build-Profiles: <!nogir>.
Or, if you're using this build profile in the mode where it changes the
contents of the libfoo-dev binary package (which is non-ideal, but avoids
needing to go through NEW), then there isn't really a recommended way,
but dh-exec is probably the least-bad way.
smcv
Helmut Grohne
Jan 24, 2024, 4:50:05 PM1/24/24
to
On Wed, Jan 24, 2024 at 06:30:02PM +0000, Alberto Garcia wrote:
> - Are packages that ship gobject-introspection files supposed to have
> <!nogir> in the relevant build dependencies (gir1.2-*-dev,
> gobject-introspection ?), or is the build profile handling this
> automatically?
This is not automatic. Please annotate relevant Build-Depends manually.
> - Are packages that ship gobject-introspection files supposed to have
> <!nogir> in the relevant build dependencies (gir1.2-*-dev,
> gobject-introspection ?), or is the build profile handling this
> automatically?
> - Packages using dh_install may have a line with usr/share/gir-1.0 in
> their debian/libfoo-dev.install. This will fail if the .gir files
> are not generated. What's the recommended way to handle this?
* You may use dh-exec. When doing so, you may annotate lines with build
profiles. For example, samba's debian/winbind.install uses this
approach.
* You may conditionally run dh_install from debian/rules passing
affected files as arguments.
* You may split the affected files into a separate binary package to
avoid this annoyance.
Helmut
0 new messages