lto-wrapper: fatal error: cc returned 1 exit status trying to build bup in Cent OS 8

[goahead97 goahead97]

Hello

I tried to build bup source files in Cent OS 8 and I got this error:

cc -I/home/x/anaconda3/include/python3.8 -I/home/x/anaconda3/include/python3.8 -Wno-unused-result -Wsign-compare -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O3 -ffunction-sections -pipe -isystem /home/x/anaconda3/include -fdebug-prefix-map=/tmp/build/80754af9/python_1599203911753/work=/usr/local/src/conda/python-3.8.5 -fdebug-prefix-map=/home/x/anaconda3=/usr/local/src/conda-prefix -fuse-linker-plugin -ffat-lto-objects -flto-partition=none -flto -DNDEBUG -fwrapv -O3 -Wall -Wno-unused-command-line-argument -D_FILE_OFFSET_BITS=64 -Wno-unknown-pragmas -Wsign-compare -O2 -Wall -Werror -Wformat=2 -I/home/x/Documents/Phd/github_repositories/bup/src -D BUP_DEV_BUP_EXEC=1 lib/cmd/bup.c src/bup/compat.c src/bup/io.c -L/home/x/anaconda3/lib/python3.8/config-3.8-x86_64-linux-gnu -L/home/x/anaconda3/lib -lpython3.8 -lcrypt -lpthread -ldl -lutil -lrt -lm -lm   -fPIE -I src -MMD -MP -o dev/bup-exec
lto1: fatal error: bytecode stream in file ‘/home/x/anaconda3/lib/python3.8/config-3.8-x86_64-linux-gnu/libpython3.8.a’ generated with LTO version 6.0 instead of the expected 7.3
compilation terminated.
lto-wrapper: fatal error: cc returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
make: *** [GNUmakefile:181: dev/bup-exec] Error 1

Does anyone know how to fix this?

Otherwise, does anyone know where I can find binary files of bup for Cent OS 8?

Thanks

[Rob Browning]

goahead97 goahead97 <goahe...@gmail.com> writes:

> Does anyone know how to fix this?
> Otherwise, does anyone know where I can find binary files of bup for Cent
> OS 8?

Not sure -- haven't had a lot of experience with LTO yet. Searching on
duckduckgo suggests that it might be because your python (or something)
was built with a compiler(?) version that's incompatible with the one
you're using now. Uncertain.

In any case, if that is the problem, it sounded like -fno-lto might
"fix" it:

make CFLAGS=-fno-lto check

(I thought we captured an existing CFLAGS at ./configure time, but looks
like we don't. Something for later.)

But whether that will work depends on whether the problematic flags end
up in the python (or whatever) cflags, or the ldflags. In the latter
case, -fno-lto might come too early on the compiler command line. If
so, you could I suppose (as a hack) try editing the compiler command(s)
in the GNUmakefile until we have a better solution.

--
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[Rob Browning]

Rob Browning <r...@defaultvalue.org> writes:

> In any case, if that is the problem, it sounded like -fno-lto might
> "fix" it:
>
> make CFLAGS=-fno-lto check

Oh, and if it is the compiler version, if you have the right version
installed, and if our makefiles are careful enough, then you might be
able to select that instead, i.e. via

CC=/usr/bin/gcc-6 ./configure

...if, for example, that error message is saying you need gcc-6 because
your python was compiled with gcc 6. Though that's just a guess.

[Greg Troxel]

for what it's worth, postgis (unrelated to bup) just published an alpha
were lto is disabled by default (and thus opt in) because of reports of
it not working right on various platforms.

[Rob Browning]

Thanks, that's good to know. Perhaps we'll end up wanting to consider
adding -fno-lto to our default flags, overridable via CFLAGS, or
similar.

Or I suppose if we were able to detect the problem in ./configure, we
might only add it when it prevents trouble.

[Greg Troxel]

Rob Browning <r...@defaultvalue.org> writes:

> Greg Troxel <g...@lexort.com> writes:
>
>> for what it's worth, postgis (unrelated to bup) just published an alpha
>> were lto is disabled by default (and thus opt in) because of reports of
>> it not working right on various platforms.
>
> Thanks, that's good to know. Perhaps we'll end up wanting to consider
> adding -fno-lto to our default flags, overridable via CFLAGS, or
> similar.

I would hope that just not adding any lto flags would avoid lto and that
would be fine. Unless CentOS 8 is lto for thw whole system?

> Or I suppose if we were able to detect the problem in ./configure, we
> might only add it when it prevents trouble.

Can somebody explain the problem, from the beginning?
Is bup intending to add lto flags?

[Rob Browning]

Greg Troxel <g...@lexort.com> writes:

> I would hope that just not adding any lto flags would avoid lto and that
> would be fine. Unless CentOS 8 is lto for thw whole system?

Right, I'm assuming it's coming in via the system's pythonX.Y-config
--cflags (guessing the most likely candidate).

> Can somebody explain the problem, from the beginning?

Build broke on CentOS 8 is about as much as I know, other than what's in
the thread so far.

> Is bup intending to add lto flags?

Nope, not at the moment.

Thanks

[Greg Troxel]

Rob Browning <r...@defaultvalue.org> writes:

> Greg Troxel <g...@lexort.com> writes:
>
>> I would hope that just not adding any lto flags would avoid lto and that
>> would be fine. Unless CentOS 8 is lto for thw whole system?
>
> Right, I'm assuming it's coming in via the system's pythonX.Y-config
> --cflags (guessing the most likely candidate).

Wow, that seems incredibly aggressive to turn on lto, especially in an
LTS release. I guess the OP should follow up with Centos and see if
lto is on system-wide, or just python, or ? and what the grand plan is.

>> Is bup intending to add lto flags?
>
> Nope, not at the moment.

Glad to hear it. (I think not getting into adding lto is the right
call.)

[Nix]

On 23 May 2022, Rob Browning verbalised:

> Greg Troxel <g...@lexort.com> writes:
>
>> for what it's worth, postgis (unrelated to bup) just published an alpha
>> were lto is disabled by default (and thus opt in) because of reports of
>> it not working right on various platforms.
>
> Thanks, that's good to know. Perhaps we'll end up wanting to consider
> adding -fno-lto to our default flags, overridable via CFLAGS, or
> similar.

I'd do that only if libpython is an .a file. If it's an .so, using LTO
should be fine, I'd think.

--
NULL && (void)

[Rob Browning]

Nix <n...@esperi.org.uk> writes:

> I'd do that only if libpython is an .a file. If it's an .so, using LTO
> should be fine, I'd think.

Hmm, I don't really know enough yet about gcc's lto to know what the
rules are, and also, for now, not sure I want to add much complexity to
accommodate this.

For the moment, the compilation related changes I just pushed may
mean that people affected can just to this:

LDFLAGS=-fno-lto ./configure && make check

but that won't be available until the next release. For the time being,
assuming we actually manage to get the value in the right places in 0.32
(at the end), people affected should be able to specify it to every make
invocation:

make LDFLAGS=-fno-lto check

goahead97, if you're still pursuing this, and that ends up working for
you, please let us know and I can consider adding something to the
README.

[Nix]

On 25 Jun 2022, Rob Browning spake thusly:

> Nix <n...@esperi.org.uk> writes:
>
>> I'd do that only if libpython is an .a file. If it's an .so, using LTO
>> should be fine, I'd think.
>
> Hmm, I don't really know enough yet about gcc's lto to know what the
> rules are, and also, for now, not sure I want to add much complexity to
> accommodate this.

The rules are fairly simple: when linking LTO-generated object files
together, they all need to be generated with compatible compilers
(usually, the same version of GCC): but only object files and .a's
containing them. The LTO version bumped a few times, as the version
number shows, and not all LTOs are intercompatible.

So for things like libpython.a, which might be generated by God knows
what GCC version, passing in -fno-lto makes sense. But LTO does not
traverse .so boundaries, so if libpython is a .so there is no danger of
traversing into it and finding object files there built by a much older
compiler. (Of course, if there are *other* .a files on the system that
have been built with LTO and that get picked up by the same link, the
same problem can recur.)

> For the moment, the compilation related changes I just pushed may
> mean that people affected can just to this:
>
> LDFLAGS=-fno-lto ./configure && make check

Seems sensible, particularly if the unaffected majority still get LTO if
they want it.

(Not that LTO is likely to do bup much good anyway, since it spends all
its time in a few hotspots in single TUs, in libpython, and in disk
I/O. More evidence your fix is right.)

--
NULL && (void)