Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Transition proposal: pkg-config to pkgconf

544 views
Skip to first unread message

Andrej Shadura

unread,
Jul 24, 2022, 11:40:03 AM7/24/22
to
Hi,

Following a discussion at DebConf, I’d like to officially propose a transition
from pkg-config to pkgconf in Debian.

pkgconf is a newer, actively maintained implementation of pkg-config that
supports more aspects of the pkg-config file specification and provides a
library interface that applications can use to incorporate intelligent
handling of pkg-config files into themselves (such as build file generators,
IDEs, and compilers).

pkgconf is compatible with pkg-config when it is run as "pkg-config", so it
can completely replace the original implementation.

Debian would benefit from switching to pkgconf by utilising a more complete
implementation of pkg-config that handles aspects of .pc files that pkg-config
does not. For example, Provides rules and Cflags.private rules are supported,
and pkgconf has a better (and less costly) dependency resolver for .pc files.

Having a more complete implementation of the pkg-config implementation can
also help convince upstreams like Postgres and LLVM to switch to using
pkg-config instead of their custom scripts like llvm-config.

pkgconf also supports a feature called personalities [1], that can be used to
implement cross-build support without a wrapper script (as currently
implemented in both pkgconf and pkg-config packages).

pkgconf is already used by other major distributions like Fedora, Gentoo and
Arch, so by doing the switch we’ll be aligning ourselves with other
distributions ([2], [3], [4])

Migration plan
==============

I believe that it would in the best interests of Debian to only ship one
pkg-config implementation, so I propose to make pkgconf provide the binary
package pkg-config without a possibility to fall back to the original
implementation. This means that after pkgconf takes over the pkg-config
binary package, all packages depending on pkg-config will be using the pkgconf
implementation without being able to opt out and use the one from
freedesktop.org. While this may seem limiting, it reduces the probability of
some packages staying with the freedesktop.org implementation indefinitely and
missing out on bug fixes or suffering from other differences in implementation
details in future when pkgconf becomes even more widespread.

In preparation for this migration, I ran a rebuild of all packages depending
on pkg-config and found only very few failures [5], which may or may not be
related to the difference in behaviour between pkg-config and pkgconf. I will
perform another rebuild and continue investigating them to provide patches
during the transition period.

[0]: http://pkgconf.org/features.html
[1]: https://manpages.debian.org/unstable/pkgconf/pkgconf-personality.5.en.html
[2]: https://fedoraproject.org/wiki/Changes/pkgconf_as_system_pkg-config_implementation
[3]: https://packages.gentoo.org/packages/virtual/pkgconfig/dependencies
[4]: https://archlinux.org/packages/core/x86_64/pkgconf/
[5]: http://pkgconf-migration.debian.net/failed.html

--
Cheers,
Andrej

Andrej Shadura

unread,
Aug 29, 2022, 9:40:02 AM8/29/22
to
Hi,

It seems like the proposal went mostly unnoticed. Any comments, ideas, anything?

Most importantly, I need a green light from the current pkg-config maintainer (Tollef) to proceed with the plan.

Meanwhile, I have uploaded an updated version of the package which dropped the crosswrapper, and instead uses the "personalities" feature to implement its functionality. The package also no longer needs a dpkg hook, as it ships a per-architecture symlinks in arch-specific packages.
--
Cheers,
Andrej

Guillem Jover

unread,
Aug 29, 2022, 10:50:04 AM8/29/22
to
Hi!

On Mon, 2022-08-29 at 15:38:08 +0200, Andrej Shadura wrote:
> It seems like the proposal went mostly unnoticed. Any comments,
> ideas, anything?

I at least read it, and it looked great. But…

> Most importantly, I need a green light from the current pkg-config
> maintainer (Tollef) to proceed with the plan.

…as it was waiting for this, it slipped my mind.

> Meanwhile, I have uploaded an updated version of the package which
> dropped the crosswrapper, and instead uses the "personalities"
> feature to implement its functionality. The package also no longer
> needs a dpkg hook, as it ships a per-architecture symlinks in
> arch-specific packages.

Thank you for getting rid of both of these annoyances! I switched all
my hosts and chroots to pkgconf, when I watched your DebConf presentation
and it's been working great.

> On Sun, 24 Jul 2022, at 17:33, Andrej Shadura wrote:
> > Following a discussion at DebConf, I’d like to officially propose a transition
> > from pkg-config to pkgconf in Debian.
> >
> > pkgconf is a newer, actively maintained implementation of pkg-config that
> > supports more aspects of the pkg-config file specification and provides a
> > library interface that applications can use to incorporate intelligent
> > handling of pkg-config files into themselves (such as build file generators,
> > IDEs, and compilers).

Right, I track the pkg-config upstream git repo and it unfortunately has
seen no major development since 2019, and not upstream release since 2017.

I've also taken a peek to pkgconf from time to time, and it looks like
a nice project, code wise, etc.

> > Debian would benefit from switching to pkgconf by utilising a more complete
> > implementation of pkg-config that handles aspects of .pc files that pkg-config
> > does not. For example, Provides rules and Cflags.private rules are supported,
> > and pkgconf has a better (and less costly) dependency resolver for .pc files.

Yes.

> > Migration plan
> > ==============
> >
> > I believe that it would in the best interests of Debian to only ship one
> > pkg-config implementation, so I propose to make pkgconf provide the binary
> > package pkg-config without a possibility to fall back to the original
> > implementation. This means that after pkgconf takes over the pkg-config
> > binary package, all packages depending on pkg-config will be using the pkgconf
> > implementation without being able to opt out and use the one from
> > freedesktop.org. While this may seem limiting, it reduces the probability of
> > some packages staying with the freedesktop.org implementation indefinitely and
> > missing out on bug fixes or suffering from other differences in implementation
> > details in future when pkgconf becomes even more widespread.

If pkg-config was showing signs of activity, then this could perhaps
be a potential concern, but given its current state, I think this makes
perfect sense.

> > In preparation for this migration, I ran a rebuild of all packages depending
> > on pkg-config and found only very few failures [5], which may or may not be
> > related to the difference in behaviour between pkg-config and pkgconf. I will
> > perform another rebuild and continue investigating them to provide patches
> > during the transition period.

Great!

Thanks,
Guillem

nick black

unread,
Aug 31, 2022, 7:10:03 AM8/31/22
to
Andrej Shadura left as an exercise for the reader:
> It seems like the proposal went mostly unnoticed. Any comments, ideas, anything?

fwiw, every time i've looked at a difference between the two,
pkgconf was superior in every way. i've worked with the primary
author, ariadne conill, on some Alpine stuff and some Notcurses
stuff, and she's as competent and sedulous as they come. i'd
love to see this transition, and appreciate you working on it.

--
nick black -=- https://www.nick-black.com
to make an apple pie from scratch,
you need first invent a universe.

Wookey

unread,
Aug 31, 2022, 8:50:03 AM8/31/22
to
On 2022-08-29 15:38 +0200, Andrej Shadura wrote:
> Hi,
>
> It seems like the proposal went mostly unnoticed. Any comments, ideas, anything?

I have an interest as having been responsible for the pkg-config
crosswrapper stuff some years ago, but have not been taking much notice
recently.

If everything you said in your mail is true (the new one is a superset
and better in every way, and the cross stuff will still work but in a
less complicated way) then it sounds like a sensible plan. I must
admit to having never heard of pkgconf before today, but you are a
sensible chap so I assume this is indeed all as you say it is.

> Meanwhile, I have uploaded an updated version of the package which dropped the crosswrapper, and instead uses the "personalities" feature to implement its functionality. The package also no longer needs a dpkg hook, as it ships a per-architecture symlinks in arch-specific packages.

I would like to understand how the cross stuff works with this
personalities thing, but it may be a while before I get round to
it. Again I shall assume that you know what you are doing :-)
If Helmut is happy then I'm sure it's good.

Wookey
--
Principal hats: Debian, Wookware, ARM
http://wookware.org/
signature.asc

Andrej Shadura

unread,
Aug 31, 2022, 11:00:03 AM8/31/22
to
Hi,

On Wed, 31 Aug 2022, at 14:41, Wookey wrote:
>> Meanwhile, I have uploaded an updated version of the package which dropped the crosswrapper, and instead uses the "personalities" feature to implement its functionality. The package also no longer needs a dpkg hook, as it ships a per-architecture symlinks in arch-specific packages.

> I would like to understand how the cross stuff works with this
> personalities thing, but it may be a while before I get round to
> it. Again I shall assume that you know what you are doing :-)
> If Helmut is happy then I'm sure it's good.

The idea is that e.g. pkgconf:amd64 provides /usr/bin/x86_64-linux-gnu-pkg-config, which is a symlink to pkgconf, and /usr/share/pkgconfig/personality.d/x86_64-linux-gnu.personality, which defines DefaultSearchPaths and SystemLibraryPaths to be appropriate values for amd64. pkgconf:amd64 itself doesn’t ship a pkgconf binary, which is, instead, provided in pkgconf-bin, a Multi-Arch: foreign package.
pkgconf detects that it’s run using a symlink, and uses it to select the matching personality, and then provides results based on these settings.
So, unless I’ve messed up with the multi-arch stuff, it should be enough to install pkgconf:<arch> to cross-build packages.

--
Cheers,
Andrej

Simon McVittie

unread,
Aug 31, 2022, 4:30:02 PM8/31/22
to
On Wed, 31 Aug 2022 at 16:53:48 +0200, Andrej Shadura wrote:
> The idea is that e.g. pkgconf:amd64 provides
> /usr/bin/x86_64-linux-gnu-pkg-config, which is a symlink to pkgconf,
> and /usr/share/pkgconfig/personality.d/x86_64-linux-gnu.personality,
> which defines DefaultSearchPaths and SystemLibraryPaths to be appropriate
> values for amd64.

Beware that to be compatible with what pkg-config does (and what
PKG_CHECK_MODULES wants), the cross-tools need to be prefixed with a
GNU tuple, and *not* a multiarch tuple. So in particular for i386, you
will need to provide /usr/bin/i686-linux-gnu-pkg-config (and presumably
a matching personality).

In other words, this is ${DEB_HOST_GNU_TYPE}-pkg-config and not
${DEB_HOST_MULTIARCH}-pkg-config. This matches the pattern used for
other cross-tools like compilers.

You might want to *also* provide ${DEB_HOST_MULTIARCH}-pkg-config, if they
differ, but it is ${DEB_HOST_GNU_TYPE}-pkg-config that is canonically used.

smcv

Andrej Shadura

unread,
Aug 31, 2022, 4:50:03 PM8/31/22
to
Hi,

On Wed, 31 Aug 2022, at 22:25, Simon McVittie wrote:
> Beware that to be compatible with what pkg-config does (and what
> PKG_CHECK_MODULES wants), the cross-tools need to be prefixed with a
> GNU tuple, and *not* a multiarch tuple. So in particular for i386, you
> will need to provide /usr/bin/i686-linux-gnu-pkg-config (and presumably
> a matching personality).
>
> In other words, this is ${DEB_HOST_GNU_TYPE}-pkg-config and not
> ${DEB_HOST_MULTIARCH}-pkg-config. This matches the pattern used for
> other cross-tools like compilers.
>
> You might want to *also* provide ${DEB_HOST_MULTIARCH}-pkg-config, if they
> differ, but it is ${DEB_HOST_GNU_TYPE}-pkg-config that is canonically used.

Thanks for pointing out, that’s something I need to fix.

--
Cheers,
Andrej

Tollef Fog Heen

unread,
Sep 25, 2022, 3:40:03 PM9/25/22
to
]] "Andrej Shadura"

> Most importantly, I need a green light from the current pkg-config
> maintainer (Tollef) to proceed with the plan.

My interest in pkg-config has waned over the years (and so has the hours
put into maintenance of it), so if you want to bring it forward, go
ahead.

Cheers,
--
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are

Andrej Shadura

unread,
Oct 20, 2022, 6:30:03 AM10/20/22
to
Hi all,

I’ve been rebuilding packages with pkgconf for the past couple of weeks, and it looks very good so far:

http://pkgconf-migration.debian.net/

I have identified and resolved some issues, and most of the build failures I’ve seen were not related to pkgconf itself, but were caused by external factors and were usually present when packages were built as usual (missing build dependencies, compiler errors etc).

The version of pkgconf package providing the pkg-config binary package has been sitting in experimental for some time. I think I have tested the upgrade process quite extensively, but it would still be great if someone else could have a look. In any case, my plan is to upload it to unstable in about two weeks time. I will then commence another round of rebuilds; meanwhile I will be working on fixing issues I’ve already run into:

https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=andrewsh%40debian.org&tag=pkgconf-rebuild-ftbfs

--
Cheers,
Andrej

Andrej Shadura

unread,
Oct 20, 2022, 6:40:03 AM10/20/22
to
Hi,

On Thu, 20 Oct 2022, at 11:32, Sebastiaan Couwenberg wrote:
> On 10/20/22 12:25, Andrej Shadura wrote:
>> I’ve been rebuilding packages with pkgconf for the past couple of weeks, and it looks very good so far:
>>
>> http://pkgconf-migration.debian.net/
>
> The "Failures only" page is empty.

Thanks, fixed.

--
Cheers,
Andrej

Sebastiaan Couwenberg

unread,
Oct 20, 2022, 6:40:03 AM10/20/22
to
On 10/20/22 12:25, Andrej Shadura wrote:
> I’ve been rebuilding packages with pkgconf for the past couple of weeks, and it looks very good so far:
>
> http://pkgconf-migration.debian.net/

The "Failures only" page is empty.

Kind Regards,

Bas

--
GPG Key ID: 4096R/6750F10AE88D4AF1
Fingerprint: 8182 DE41 7056 408D 6146 50D1 6750 F10A E88D 4AF1

Johannes Schauer Marin Rodrigues

unread,
Oct 20, 2022, 7:01:26 AM10/20/22
to
Quoting Andrej Shadura (2022-10-20 12:25:13)
> I’ve been rebuilding packages with pkgconf for the past couple of weeks, and
> it looks very good so far:
>
> http://pkgconf-migration.debian.net/

Thank you! Attached is a dd-list of those packages listed in the "Failures
only" page in case somebody wants to have a quick glance whether their package
is affected.

Thanks!

cheers, josch
pkgconf.ddlist
signature.asc

Sebastiaan Couwenberg

unread,
Oct 20, 2022, 7:40:03 AM10/20/22
to
On 10/20/22 12:25, Andrej Shadura wrote:
> The version of pkgconf package providing the pkg-config binary package has been sitting in experimental for some time. I think I have tested the upgrade process quite extensively, but it would still be great if someone else could have a look. In any case, my plan is to upload it to unstable in about two weeks time. I will then commence another round of rebuilds; meanwhile I will be working on fixing issues I’ve already run into:

I cannot reproduce the issue with icinga2, it built successfully on my
system with pkg-config (>= 1.8.0) from experimental.

Andrej Shadura

unread,
Oct 20, 2022, 8:30:03 AM10/20/22
to
Hi,

On Thu, 20 Oct 2022, at 12:34, Sebastiaan Couwenberg wrote:
> On 10/20/22 12:25, Andrej Shadura wrote:
>> The version of pkgconf package providing the pkg-config binary package has been sitting in experimental for some time. I think I have tested the upgrade process quite extensively, but it would still be great if someone else could have a look. In any case, my plan is to upload it to unstable in about two weeks time. I will then commence another round of rebuilds; meanwhile I will be working on fixing issues I’ve already run into:
>
> I cannot reproduce the issue with icinga2, it built successfully on my
> system with pkg-config (>= 1.8.0) from experimental.

Yes, this one in particular looks like an OOM error. I will do a separate run with more memory and disk just for those ABORTED builds.

--
Cheers,
Andrej

Scott Talbert

unread,
Oct 20, 2022, 9:10:04 AM10/20/22
to
Also, it looks like there are some transient failures due to the perl
rebuild that's ongoing. Specifically wxwidgets3.0 and wxpython4.0 are
affected by that.

Scott

Andrej Shadura

unread,
Oct 20, 2022, 9:20:02 AM10/20/22
to
Hi,

On Thu, 20 Oct 2022, at 14:03, Scott Talbert wrote:
> Also, it looks like there are some transient failures due to the perl
> rebuild that's ongoing. Specifically wxwidgets3.0 and wxpython4.0 are
> affected by that.

Yes, that’s why I added a log parser yesterday (thanks, Jelmer!) to try and detect those (it adds an annotation e.g. "unsatisfied-apt-dependencies" to the FAILED).

--
Cheers,
Andrej

Leopold Palomo-Avellaneda

unread,
Oct 20, 2022, 4:10:03 PM10/20/22
to
El 20/10/22 a les 12:56, Johannes Schauer Marin Rodrigues ha escrit:
I have rebuilt fcl changing pkg-config by pkgconf in Build-dependencies
in a pbuilder environment and it has been successful. So, I guess that
it can been dropped from failure list.

Thanks for the work Andrej,


Leopold

--
--
Linux User 152692 GPG: 05F4A7A949A2D9AA
Catalonia
-------------------------------------
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?

Nilesh Patra

unread,
Oct 20, 2022, 4:20:03 PM10/20/22
to
Hi,

On Thu, Oct 20, 2022 at 11:25:13AM +0100, Andrej Shadura wrote:
> The version of pkgconf package providing the pkg-config binary package has been sitting in experimental for some time. I think I have tested the upgrade process quite extensively, but it would still be great if someone else could have a look. In any case, my plan is to upload it to unstable in about two weeks time. I will then commence another round of rebuilds; meanwhile I will be working on fixing issues I’ve already run into:
>
> https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=andrewsh%40debian.org&tag=pkgconf-rebuild-ftbfs

Since jellyfish is listed at that link, I tried to dig further and it "seems" that pkgconf
does not honor PKG_CONFIG_SYSROOT_DIR. More details at #1022087

--
Best,
Nilesh
signature.asc

Andreas Metzler

unread,
Oct 21, 2022, 12:40:03 PM10/21/22
to
[...]
> Andreas Metzler <amet...@debian.org>
> enblend-enfuse (U)

Works for me. - It is now also marked with SUCCESS on your webpage, I
guess you retried?

cu Andreas

Andrej Shadura

unread,
Oct 21, 2022, 1:10:03 PM10/21/22
to
Hi,

On Fri, 21 Oct 2022, at 18:38, Andreas Metzler wrote:
>> Andreas Metzler <amet...@debian.org>
>> enblend-enfuse (U)
>
> Works for me. - It is now also marked with SUCCESS on your webpage, I
> guess you retried?

I guess so :)

--
Cheers,
Andrej

Andrej Shadura

unread,
Nov 9, 2022, 7:30:03 AM11/9/22
to
Hi,

Just a small update: I’m now confident we’re ready to go ahead with the actual transition.
I’m still rebuilding some heavy packages that failed to build because of lack of disk space or RAM, but most of the other failures are unrelated to pkgconf, while remaining issues can be resolved later. I will perform a deeper analysis of the failures and will file bugs where appropriate.

I will upload pkgconf = 1.8.0-10 + pkg-config = 1.8.0-10 this Friday, 2022-11-11.

[1]: http://pkgconf-migration.debian.net/
[2]: https://udd.debian.org/cgi-bin/bts-usertags.cgi?user=andrewsh%40debian.org&tag=pkgconf-rebuild-ftbfs

--
Cheers,
Andrej
0 new messages