[llvm-dev] Should lld support binary output ("--oformat binary")?

[Ed Maste via llvm-dev]

One remaining blocking issue for linking FreeBSD with lld is the
inability to build the bootloader components. We have a patch[1] in
review to address one of the problems by switching to using a linker
script instead of GNU ld's -N option. Another issue is that some
bootloader components are created by the linker directly as binary
objects, using --oformat binary. (Other bootloader components are
built by creating a temporary ELF object converting that to a raw
binary using objcopy.)

Is binary output enough of an unusual use case that we're unlikely to
support it in lld?

[1] https://reviews.freebsd.org/D7409
_______________________________________________
LLVM Developers mailing list
llvm...@lists.llvm.org
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

[Davide Italiano via llvm-dev]

On Thu, Aug 18, 2016 at 2:16 PM, Ed Maste via llvm-dev
<llvm...@lists.llvm.org> wrote:
> One remaining blocking issue for linking FreeBSD with lld is the
> inability to build the bootloader components. We have a patch[1] in
> review to address one of the problems by switching to using a linker
> script instead of GNU ld's -N option. Another issue is that some
> bootloader components are created by the linker directly as binary
> objects, using --oformat binary. (Other bootloader components are
> built by creating a temporary ELF object converting that to a raw
> binary using objcopy.)
>
> Is binary output enough of an unusual use case that we're unlikely to
> support it in lld?
>

On a related note. FreeBSD (and also a product on which I work on) uses
-b for kernel modules. I really would like to avoid changing all the
invocations to objcopy. Hw vendors (e.g. network drivers which ship
with firmware) rely on this feature. With that in mind, I'm not
exactly looking forward to make them unhappy and force to change their
internal build, in particular considering how hard is to get driver
support these days.

--
Davide

"There are no solved problems; there are only problems that are more
or less solved" -- Henri Poincare

[Rui Ueyama via llvm-dev]

Maybe we should implement it. I had a concern about adding complexity for the less frequently used feature, but I believe it can be implemented easily. Specifically, I think we need to

 - implement another version of `Writer<ELFT>::writeSections` for --oformat=binary and

 - stop calling `Writer::writeHeaders` and `writeBuildId` if --oformat=binary is specified.

[Sean Silva via llvm-dev]

On Thu, Aug 18, 2016 at 8:32 AM, Davide Italiano via llvm-dev <llvm...@lists.llvm.org> wrote:

On Thu, Aug 18, 2016 at 2:16 PM, Ed Maste via llvm-dev
<llvm...@lists.llvm.org> wrote:
> One remaining blocking issue for linking FreeBSD with lld is the
> inability to build the bootloader components. We have a patch[1] in
> review to address one of the problems by switching to using a linker
> script instead of GNU ld's -N option. Another issue is that some
> bootloader components are created by the linker directly as binary
> objects, using --oformat binary. (Other bootloader components are
> built by creating a temporary ELF object converting that to a raw
> binary using objcopy.)
>
> Is binary output enough of an unusual use case that we're unlikely to
> support it in lld?
>

On a related note. FreeBSD (and also a product on which I work on) uses
-b for kernel modules. I really would like to avoid changing all the
invocations to objcopy. Hw vendors (e.g. network drivers which ship
with firmware) rely on this feature. With that in mind, I'm not
exactly looking forward to make them unhappy and force to change their
internal build, in particular considering how hard is to get driver
support these days.

This is the next issue that Michael and I ran into after figuring out what was needed to get the kernel to boot (the remaining patches for that are in review).

It turns out that implementing that is very simple (essentially identical to the shader binary stuff that we already implemented for PS4, which internally uses the same mechanism). Yesterday Michael ported that code onto ToT and hopefully there will be a patch up soon.

-- Sean Silva

[Rui Ueyama via llvm-dev]

As long as it's simple (easy to implement) and non-intrusive (doesn't touch a lot of places in the code base), we should support that too.

[George Rimar via llvm-dev]

> Maybe we should implement it. I had a concern about adding complexity for
> the less frequently used feature, but I believe it can be implemented
> easily. Specifically, I think we need to
>
> - implement another version of `Writer<ELFT>::writeSections` for
> --oformat=binary and
> - stop calling `Writer::writeHeaders` and `writeBuildId` if
> --oformat=binary is specified.

Patch implementing --oformat binary is here: https://reviews.llvm.org/D23769
I noticed that ld and gold has idfferent behavior:
gold uses LMA of PT_LOADs to find the destination file offset of data which it takes from segments.
I tried to play with AT() and looks GNU ld ignores that.

Patch implements ld behavior for now.

George.