Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Bug#1033015: Minor issues in debian/rules [patch]

362 views
Skip to first unread message

Daniel Richard G.

unread,
Mar 15, 2023, 3:02:28 PM3/15/23
to
Package: chromium
Version: 111.0.5563.64-1
Severity: minor

I've been experimenting with building the latest Debian chromium package
on Ubuntu 22.04, and in the course of that have found a couple of minor
issues in the debian/rules file:

* The flags added by optimize=+lto in DEB_BUILD_MAINT_OPTIONS will
cause every compile invocation to print "optimization flag
'-ffat-lto-objects' is not supported" warnings (as the dpkg LTO flags
are meant for GCC, not Clang), and balloon the RAM required for the
final link from ~2.5 GB to ~30 GB. The link will also require much
more time to complete; on my fairly beefy test system, it went from
under a minute to four hours.

Disabling this explicitly before Debian enables LTO system-wide would
be a good move, in my view. On Ubuntu, LTO is already the default, and
without adding this bit, the package is difficult to build in their
infrastructure due to the resource requirements.

* LDFLAGS is set without obtaining an initial value from
dpkg-buildflags(1).

The attached patch addresses both issues.

Side note: You may want to consider enabling ThinLTO, by setting
use_thin_lto=true and unsetting concurrent_links. The final link
requires only ~10.5 GB RAM, and completes within minutes.


--Daniel


--
Daniel Richard G. || sk...@iSKUNK.ORG
My ASCII-art .sig got a bad case of Times New Roman.
rules.patch

Daniel Richard G.

unread,
Mar 17, 2023, 7:00:04 AM3/17/23
to
An addendum:

The package currently builds at debhelper compat level 11. If this is
increased to 13, the build fails at the dh_dwz step. Build log excerpts
are below.

The build uses symbol_level=0 in debian/rules (which causes -g0 to be
passed to the compiler). If this is set to 1, dh_dwz succeeds. (If this
is set to 2, dh_dwz fails with a different error.)

Alternately, DEB_BUILD_OPTIONS=nostrip works around the issue. However,
it results in a number of "Unrecognized form" warnings from objdump(1)
and readelf(1), for some reason.

(I hope this is useful information, as this issue is likely to crop up
in the Debian build sooner or later.)


Build failure with compat 13:
----begin----
dh_missing
dh_dwz -a
dwz -- debian/chromium-driver/usr/bin/chromedriver
dwz -- debian/chromium-shell/usr/lib/chromium/chromium-shell
dwz -- debian/chromium-sandbox/usr/lib/chromium/chrome-sandbox
dwz: debian/chromium-shell/usr/lib/chromium/chromium-shell: Unknown debugging section .debug_str_offsets
dwz: debian/chromium-driver/usr/bin/chromedriver: .debug_info section not present
dh_dwz: error: dwz -- debian/chromium-shell/usr/lib/chromium/chromium-shell returned exit code 1
dwz: debian/chromium-sandbox/usr/lib/chromium/chrome-sandbox: Unknown debugging section .debug_str_offsets
dwz -- debian/chromium/usr/lib/chromium/chromium
dh_dwz: error: dwz -- debian/chromium-sandbox/usr/lib/chromium/chrome-sandbox returned exit code 1
dwz: debian/chromium/usr/lib/chromium/chromium: Unknown debugging section .debug_str_offsets
install -d debian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu
dwz -mdebian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -M/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -- debian/chromium-common/usr/lib/chromium/chrome_crashpad_handler debian/chromium-common/usr/lib/chromium/libEGL.so debian/chromium-common/usr/lib/chromium/libGLESv2.so
dh_dwz: error: dwz -- debian/chromium/usr/lib/chromium/chromium returned exit code 1
dwz: debian/chromium-common/usr/lib/chromium/chrome_crashpad_handler: Unknown debugging section .debug_str_offsets
dwz: debian/chromium-common/usr/lib/chromium/libEGL.so: Unknown debugging section .debug_str_offsets
dwz: debian/chromium-common/usr/lib/chromium/libGLESv2.so: Unknown debugging section .debug_str_offsets
dwz: Too few files for multifile optimization
dh_dwz: error: dwz -mdebian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -M/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -- debian/chromium-common/usr/lib/chromium/chrome_crashpad_handler debian/chromium-common/usr/lib/chromium/libEGL.so debian/chromium-common/usr/lib/chromium/libGLESv2.so returned exit code 1
dh_dwz: error: Aborting due to earlier error
make: *** [debian/rules:112: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
----end----

Build success with compat 13 and symbol_level=1:
----begin----
dh_missing
dh_dwz -a
dwz -- debian/chromium-driver/usr/bin/chromedriver
dwz -- debian/chromium-sandbox/usr/lib/chromium/chrome-sandbox
dwz -- debian/chromium-shell/usr/lib/chromium/chromium-shell
dwz: debian/chromium-driver/usr/bin/chromedriver: .debug_info section not present
dwz -- debian/chromium/usr/lib/chromium/chromium
install -d debian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu
dwz -mdebian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -M/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -- debian/chromium-common/usr/lib/chromium/chrome_crashpad_handler debian/chromium-common/usr/lib/chromium/libEGL.so debian/chromium-common/usr/lib/chromium/libGLESv2.so
objcopy --compress-debug-sections debian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug
dh_strip -a
----end----

Build failure with compat 13 and symbol_level=2:
----begin----
dh_missing
dh_dwz -a
dwz -- debian/chromium-driver/usr/bin/chromedriver
dwz -- debian/chromium-shell/usr/lib/chromium/chromium-shell
dwz -- debian/chromium-sandbox/usr/lib/chromium/chrome-sandbox
dwz: debian/chromium-driver/usr/bin/chromedriver: .debug_info section not present
dwz -- debian/chromium/usr/lib/chromium/chromium
install -d debian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu
dwz -mdebian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -M/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug -- debian/chromium-common/usr/lib/chromium/chrome_crashpad_handler debian/chromium-common/usr/lib/chromium/libEGL.so debian/chromium-common/usr/lib/chromium/libGLESv2.so
dwz: debian/chromium/usr/lib/chromium/chromium: Too many DIEs, not optimizing
dh_dwz: error: dwz -- debian/chromium/usr/lib/chromium/chromium returned exit code 1
dwz: debian/chromium-shell/usr/lib/chromium/chromium-shell: Too many DIEs, not optimizing
dh_dwz: error: dwz -- debian/chromium-shell/usr/lib/chromium/chromium-shell returned exit code 1
objcopy --compress-debug-sections debian/chromium-common/usr/lib/debug/.dwz/x86_64-linux-gnu/chromium-common.debug
dh_dwz: error: Aborting due to earlier error
make: *** [debian/rules:112: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
----end----

Build success with compat 13 and DEB_BUILD_OPTIONS=nostrip:
----begin----
dh_missing
dh: command-omitted: The call to "dh_dwz" was omitted due to "DEB_BUILD_OPTIONS=nostrip"
dh: command-omitted: The call to "dh_strip" was omitted due to "DEB_BUILD_OPTIONS=nostrip"
dh_makeshlibs -a
rm -f debian/chromium/DEBIAN/shlibs
objdump: Warning: Unrecognized form: 0x22
objdump: Warning: Unrecognized form: 0x22
objdump: Warning: Unrecognized form: 0x22
objdump: Warning: Unrecognized form: 0x22
----end----

Andres Salomon

unread,
Mar 17, 2023, 9:40:05 PM3/17/23
to


On Wed, Mar 15 2023 at 02:46:21 PM -04:00:00, Daniel Richard G.
<sk...@iSKUNK.ORG> wrote:
> Package: chromium
> Version: 111.0.5563.64-1
> Severity: minor
>
> I've been experimenting with building the latest Debian chromium
> package
> on Ubuntu 22.04, and in the course of that have found a couple of
> minor
> issues in the debian/rules file:
>
> * The flags added by optimize=+lto in DEB_BUILD_MAINT_OPTIONS will
> cause every compile invocation to print "optimization flag
> '-ffat-lto-objects' is not supported" warnings (as the dpkg LTO
> flags
> are meant for GCC, not Clang), and balloon the RAM required for the
> final link from ~2.5 GB to ~30 GB. The link will also require much
> more time to complete; on my fairly beefy test system, it went from
> under a minute to four hours.
>
> Disabling this explicitly before Debian enables LTO system-wide
> would
> be a good move, in my view. On Ubuntu, LTO is already the default,
> and
> without adding this bit, the package is difficult to build in their
> infrastructure due to the resource requirements.


Is Ubuntu looking to switch to using Debian's chromium instead of the
snap or flatpak or whatever they currently use?

If that's a "yes", and you're the one working on it, I'd suggest
joining Debian's chromium team. :)




>
> * LDFLAGS is set without obtaining an initial value from
> dpkg-buildflags(1).
>
> The attached patch addresses both issues.


Thanks! I'll apply the patch (though I'm not sure yet if we want more
verbose linker output, build logs are already pretty big).


>
> Side note: You may want to consider enabling ThinLTO, by setting
> use_thin_lto=true and unsetting concurrent_links. The final link
> requires only ~10.5 GB RAM, and completes within minutes.


I'm not clear on how that affects runtime performance; any ideas? I'd
also have to make sure that it works okay on clang-13 (which is what
bullseye is building with).

Daniel Richard G.

unread,
Mar 18, 2023, 3:00:04 AM3/18/23
to
On Fri, 2023 Mar 17 21:26-04:00, Andres Salomon wrote:
>
> Is Ubuntu looking to switch to using Debian's chromium instead of the
> snap or flatpak or whatever they currently use?

Unfortunately no, not to my knowledge. I'm just working with a
gentleman who maintains an Ubuntu PPA build of Chromium, that is
based on Debian's source.

> If that's a "yes", and you're the one working on it, I'd suggest
> joining Debian's chromium team. :)

That's what they should have done, all right...

>> The attached patch addresses both issues.
>
> Thanks! I'll apply the patch (though I'm not sure yet if we want more
> verbose linker output, build logs are already pretty big).

Oh, I didn't add that bit; my patch just moves it down in the file.

>> Side note: You may want to consider enabling ThinLTO, by setting
>> use_thin_lto=true and unsetting concurrent_links. The final link
>> requires only ~10.5 GB RAM, and completes within minutes.
>
> I'm not clear on how that affects runtime performance; any ideas? I'd
> also have to make sure that it works okay on clang-13 (which is what
> bullseye is building with).

I haven't run any benchmarks myself, but commentary from folks who do
performance-optimized builds of Chromium (https://thorium.rocks/ is one
example) seem to indicate that ThinLTO is beneficial.

(LTO generally is, isn't it? It's just that with Chromium, standard LTO
is prohibitively expensive.)

I tried a test build on bullseye with use_thin_lto=true, and it goes
through, FWIW. I don't have at this time an environment to actually try
out the build, however.

Daniel Richard G.

unread,
Jun 19, 2023, 10:50:04 PM6/19/23
to
In recent versions, enabling use_thin_lto causes a new linker flag to be
added, which as far as I've seen, Clang <= 15 doesn't recognize. (I have
not tested >= 16)

This is the build error I've observed:

[2789/56743] SOLINK ./libvulkan.so.1
FAILED: libvulkan.so.1 libvulkan.so.1.TOC
"python3" "../../build/toolchain/gcc_solink_wrapper.py" --readelf="readelf" --nm="nm" --sofile="./libvulkan.so.1" --tocfile="./libvulkan.so.1.TOC" --output="./libvulkan.so.1" -- clang++ -shared -Wl,-soname="libvulkan.so.1" -fuse-ld=lld -Wl,--fatal-warnings -Wl,--build-id=sha1 -fPIC -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--icf=all -Wl,--color-diagnostics -Wl,-mllvm,-instcombine-lower-dbg-declare=0 -flto=thin -Wl,--thinlto-jobs=all -Wl,--thinlto-cache-dir=thinlto-cache -Wl,--thinlto-cache-policy=cache_size=10\%:cache_size_bytes=40g:cache_size_files=100000 -Wl,-mllvm,-import-instr-limit=30 -Wl,-mllvm,-disable-auto-upgrade-debug-info -fwhole-program-vtables -Wl,--undefined-version -Wl,--no-call-graph-profile-sort -m64 -no-canonical-prefixes -Wl,-O2 -Wl,--gc-sections -rdynamic -Wl,-z,defs -Wl,--as-needed -Wl,--lto-O0 -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-z,now -Wl,--stats -o "./libvulkan.so.1" @"./libvulkan.so.1.rsp"
ld.lld: error: -mllvm: ld.lld: Unknown command line argument '-disable-auto-upgrade-debug-info'. Try: '/usr/bin/ld.lld --help'
ld.lld: Did you mean '--disable-arm-parallel-dsp'?
clang++: error: linker command failed with exit code 1 (use -v to see invocation)


The following patch fixes the issue:

--- a/build/config/compiler/BUILD.gn
+++ b/build/config/compiler/BUILD.gn
@@ -726,14 +726,6 @@ config("compiler") {
}

ldflags += [ "-Wl,-mllvm,-import-instr-limit=$import_instr_limit" ]
-
- if (!is_chromeos) {
- # TODO(https://crbug.com/972449): turn on for ChromeOS when that
- # toolchain has this flag.
- # We only use one version of LLVM within a build so there's no need to
- # upgrade debug info, which can be expensive since it runs the verifier.
- ldflags += [ "-Wl,-mllvm,-disable-auto-upgrade-debug-info" ]
- }
}

# TODO(https://crbug.com/1211155): investigate why this isn't effective on

Daniel Richard G.

unread,
Nov 19, 2023, 12:00:05 AM11/19/23
to
On Sat, 2023 Nov 18 20:37-05:00, Andres Salomon wrote:
>
> I still haven't messed around with thinlto yet, but my primary concern
> at this point is that various ARM build machines keep running out of
> memory while building chromium. In particular, on armhf the arm-ubc-*
> buildds run out of memory and kill the build,

On armhf, it's not possible to enable ThinLTO, because the final link
requires more than 4 GB RAM. (I tried a full build in an armhf Docker
container running under QEMU, on a system with 48 GB RAM, and still
it failed.) I haven't tried i386, but presumably the same issue will
occur there.

(Is there a way to have official Debian packages be cross-compiled? This
is a case where native compilation isn't ideal.)

> as you can see on
> https://buildd.debian.org/status/logs.php?pkg=chromium&arch=armhf

There's something unrelated going on there, doubtful it's any kind of
OOM at that point in the build. I think you'd have to peek at syslog to
get an answer. Could be a hardware issue, for all the log shows.

> I'm wondering if thinlto would possibly result in faster/less-memory-
> intensive object (or bitcode or whatever) generation that might work
> better on platforms with less memory? Do you know offhand how memory
> usage of thinlto compares to what we (debian) do now (no LTO at all)?

I'm not aware of LTO, in general, helping much with run-time memory
usage. But it's supposed to make things run faster to some degree, and I
figure that alone would be welcome on the less-beefy platforms.


By the way, I've been meaning to write in with a related issue. ThinLTO
currently fails on ppc64el, with this GN error:

----begin build log excerpt----
gn gen out/Release --args="clang_use_chrome_plugins=false ... use_thin_lto=true ..."
ERROR at //build/config/compiler/BUILD.gn:696:5: Assertion failed.
assert(use_lld, "LTO is only supported with lld")
^-----
LTO is only supported with lld
See //build/config/BUILDCONFIG.gn:333:3: which caused the file to be included.
"//build/config/compiler:afdo",
^-----------------------------
make[1]: *** [debian/rules:164: override_dh_auto_build-arch] Error 1
make[1]: Leaving directory '/<<PKGBUILDDIR>>'
make: *** [debian/rules:127: binary] Error 2
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
----end build log excerpt----

(Credit to Luiz Guzman @ Trisquel Linux for bringing this to my attention:
https://github.com/ungoogled-software/ungoogled-chromium-debian/issues/334#issuecomment-1767888316)

The issue comes down to this conditional in build/config/BUILDCONFIG.gn:

# Set to true when compiling with the Clang compiler.
is_clang = current_os != "linux" ||
(current_cpu != "s390x" && current_cpu != "s390" &&
current_cpu != "ppc64" && current_cpu != "ppc" &&
current_cpu != "mips" && current_cpu != "mips64" &&
current_cpu != "riscv64")

The assumption underlying that logic is clearly incorrect in the Debian
context. Perhaps we should just hard-code it to true.

I don't know if there are other ramifications of is_clang=false in the
ppc64el build. I sent a note to Tim about this, but never heard back.
0 new messages