Proposal for a new e_type value to be used for static libraries

[Eyal Itkin]

Greetings,

I have prepared the following proposal to extend the ELF standard by adding a new e_type value: ET_STAT.

The value will be used by a new proposed format (Static Bundle Object) that hopefully will be used to represent static libraries, and will replace the currently used ".a" archive of ET_REL object code (.o) files. The end goal is to overcome the limitations in the ".a" static archive format that is currently used for static libraries, and bring the static libraries format closer to the ET_DYN e_type used for dynamic libraries.

Link to the technical white paper with the proposal:

* File name: "2025-09 - Static Bundle Object - Proposal for a New ELF Type for Static Library Linking.pdf"

* Link to the repository: https://github.com/eyalitki/presentations

* Link to the PDF itself: https://github.com/eyalitki/presentations/blob/master/2025-09%20-%20Static%20Bundle%20Object%20-%20Proposal%20for%20a%20New%20ELF%20Type%20for%20Static%20Library%20Linking.pdf

The paper also refers to a proof-of-concept implementation of the suggested format on top of GNU's ld, as can be found here:

https://github.com/eyalitki/binutils-gdb/tree/static_bundle

Will be more than happy to assist in any way needed during the discussion about this topic.

Thanks in advance for your review,

Eyal Itkin

[Roland McGrath]

I've only looked at your paper briefly, but I'm not sure this offers anything over `ld -r` + `objcopy --localize-hidden` as can already be used today (or `--globalize-symbol=...`, `--keep-global-sym=...`, etc.).  How does an ET_STAT file differ from an ET_REL file whose symbols have been rewritten to be STB_LOCAL when not "exported'?  New ELF format features are not always required to use ELF in new ways.  (But AFAICT this is a fairly well-established way, not a new one--just an arcane one.)

--
You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/generic-abi/5d1201ad-0d8d-47ec-8ab6-4316e4401fean%40googlegroups.com.

[Eyal Itkin]

Thanks for the reply. Indeed in my paper I do present the option of using `ld -r`, the downsides for it and how I used it as a basis for my proposed format.

And more specifically:

1. There are helper projects (such as https://github.com/tux3/armerge) that aim to provide similar (if incomplete) support for adjusting static libraries, and indeed use a similar approach of `ld -r` and various objcopy commands as suggested.

2. The author of `armerge` is supporting my proposal and they describe it better than I can ever describe it, so I recommend reading their feedback as cited: source.

3. If my library attempts to statically link to another library, the handling is more complex (more than just "--localize-hidden"):

 a) One will need to also unpack the static library of the dependency - Unpacking has several issues due to possibly multiple occurrences of same name file (even in same archive)
 b) Using "--localize-symbol" per symbol of the statically compiled dependency is cumbersome - Projects don't usually have a list of the APIs of other projects
 c) Using "--keep-global-symbol" is quite messy given that shared libraries often use a version.map for it and now we force project to somehow juggle between these two lists of symbols

4. And finally, in both cases ("--localize-hidden" and manual hiding/keeping of symbols), we would still have a relocation tied to the internal (secret) functions, thus blocking strip from removing their symbols from the binary:
"
strip -Nsecret_function unified_lib.o
strip: not stripping symbol `secret_function' because it is named in a relocation
"

Given that objcopy does not support the option to resolve relocations related to updated symbols (local/global), the proper way to handle them (so to enable stripping the symbol) is to have ld support for the relocation resolution, which is indeed what I aim for in my proposal.

In the end, it might be possible to use the existing ET_REL e_type alongside more common/arcane ld flags and objcopy flags. However, as mentioned in my paper, I believe the correct approach is a new e_type:

1) Today ET_REL doesn't offer symbol visibility features when processed by ld - Changing that or changing the behavior of `ld -r` is a compatibility risk.

2) There are quite a few downsides for these flags (as shown above) aside from the process being far from elegant.

To summarize, my proposal aims to officially define the feature-set for a static library and have a dedicated e_type for it in the ELF format (coupled with official ld support), just as was done long ago for shared libraries (ET_DYN). 

[Roland McGrath]

There is certainly a case to be made for having better integration in tools to produce the kinds of ELF files you want.  I don't think you've done anything to make a case for any new ELF format features to enable building tools with the semantics you'd like to have.

[Eyal Itkin]

First, I'm happy we agree that there is a case to be made for better integration of tools for producing more optimal ELF files for static libraries.

As for the means of doing so, as I suggested in the paper and in this thread, the ELF file will need to either keep using the e_type of ET_REL, or use a new value (ET_STAT) that will be allocated for it. I admit I might not fully understand the cases in which an e_type value is allocated, yet I will point out there is a value of ET_DYN (shared libraries) and even a value for core dumps (which strictly speaking, are not executable files).

If the value of ET_REL will be kept, it will now serve 3 different roles, with completely different meanings and feature sets:

1) The plain compilation output - the compiler-produced object-code file

2) The plain amalgamation of multiple object-code files using the `ld-r` file - Which as discussed, involves almost no logic on behalf of the linker, which doesn't resolve relocations nor does it apply visibility semantics

3) A more complex amalgamation of multiple object-code files, that is done in the spirit of creating ET_DYN files, and that shares most of the logic with them, as shown in my proof-of-concept implementation on top of GNU's ld.

I agree this is feasible solution. However, from a compatibility standpoint changing the behavior of the existing `ld -r` flag is probably out of the question. And so, if a new linker flow is anyway being suggested in order to create a new binary ELF format, why shouldn't we assign a new e_type for it and make it official?

I do not know what are the official definitions for the exact scope of each existing e_type so to be able to decide if the new format falls within the existing scope or outside of it. If there are such, I believe it would be productive for this thread to discuss them and compare them to the requested format. In addition, I also do not know the rules by which new values are allocated, and would appreciate it if those could be shared as well given the nature of this thread.

What I do know is that there is plenty of room in the existing enum, and there is no risk of bloating the use of the e_type field with many new values, as the ask here is to align the static libraries with the attention that the dynamic libraries have already received. Given there are are only 2 types of such libraries, one of which is already covered by the standard, I do not anticipate a flood of new such e_type values if the "gates will be opened" for this specific request.

Obviously, the esteemed forum here has the final say, and any decision that you make will be appreciated and respected. When deciding about the proposal I made, I hope you will take into account the time and effort that went into preparing the paper and submitting this proposal, as well as the feedback the initial proposal received when it was circled about roughly 2 months ago in the community. I would not have taken the effort to work on it if I wouldn't have believed there are real gaps in the current tooling that require improvement, and I wouldn't have submitted my proposal to you if I wouldn't believe that this is the correct approach to help address these industry-wide issues.

Thanks again for your time,

Eyal

[connor horman]

Yeah, I'm going to strongly agree with the things above - we have plenty of e_type space, and static libraries as either `ld -r` ET_REL objects or as .a files are second class in ELF compared for first-class ET_DYN objects. I also think that we do need the support that ET_DYN gets for static libraries, looking at projects like Rust, which have a difficult time keeping internal symbols from colliding in diagnosable (or non-diagnosable) ways when you think a static system library written in Rust to another, or to a program that uses Rust.

--

You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/generic-abi/27d4228c-a831-4056-88c6-d41fb0eb2502n%40googlegroups.com.

[Fangrui Song]

There is some misinformation.

>  If different visibility attributes are specified for distinct references to or definitions of a symbol, the most constraining visibility attribute must be propagated to the resolving symbol in the linked object. The attributes, ordered from least to most constraining, are: STV_PROTECTED, STV_HIDDEN, STV_INTERNAL, and STV_EXPORTED.

In GNU ld-compatible linkers, output constrains visibility regardless of -r.
It is incorrect to state that "ET_REL doesn't offer symbol visibility features when processed by ld".
Non-relocatable outputs (ET_EXEC, ET_DYN) additionally convert symbols of hidden visibility to STB_LOCAL binding.

> Similarly to shared objects, static bundle objects will resolve relocations for local symbols, instead of keeping them as possible hook points in the finalized binary

Only a limited set of relocations can actually be resolved in relocatable files.
Specifically, this applies to relocations following the (S-P+A) formula where S refers to a non-ifunc local symbol in the same section as P.
This behavior mirrors assembler fixup resolution (https://maskray.me/blog/2025-03-16-relocation-generation-in-assemblers ):

> Fixup resolution depends on the fixup type:
>
> - PC-relative fixups that describe the symbol itself (the relocation operation looks like S - P + A) resolve to a constant if sym_a is a non-ifunc local symbol defined in the current section.
> - relocation_specifier(S + A) style fixups resolve when S refers to an absolute symbol.
> - Other fixups, including TLS and GOT related ones, remain unresolved.

Many scenarios prevent relocation resolution:

- Cross-section symbol references
- Symbols with STB_WEAK or STB_GLOBAL binding (due to potential symbol interposition in shared libraries and linker script symbol assignments)
- TLS or GOT-related relocations
- Relocations potentially requiring PLT entries or range-extension thunks

---

ET_REL object file already serve different roles.

- Compiler output (e.g. cc -c)
- Relocatable linking output - These files can be:
  * Used directly as input for executable linking (ld -r a.o b.o -o ab.o; ld ab.o)
  * Loaded by custom loaders (e.g., kernel modules in Linux)
  * Merged further with other relocatable files (ld -r a.o b.o -o ab.o; ld -r ab.o c.o -o abc.o)

The proposed static bundle object appears to cover a subset of existing ET_REL use cases while adding a new e_type code. I don't see this as a necessary addition.

---

In the current relocatable linking based approach, we should also consider COMDAT section groups when using C++, which enable name-based deduplication (binding is ignored).

- Compile with hidden symbols by default. Mark necessary symbols as exported
- Run ld -r --force-group-allocation
- Process the output with objcopy --localize-hidden with possibly other options.

objcopy --keep-global-symbols=filename can be utilized as well, similar to how we use a version scriot to create a shared object.

I agree that we might need better integration in tools (including the build system) to produce the kinds of ELF files you want.

[Rafael Ávila de Espíndola]

Hi,

First of all, thanks. The use of .a files with their different semantics always felt a bit odd.

Having said that,  I think you make a good case for a new linker command line (--static-lib?), not a new elf file type.

The options should not combine sections or remove relocations so that gc sections, icf, and section placement still work to the best of their performance.

The option should localise symbols that would be hidden if this was producing a shared library.

Basically,  a .c file with a public function that calls a static one, when compiled with -ffunction-sections, should produce the same result as one of the functions being moved to another file, made hidden, and the two object files linked with the new option.

Thanks,
Rafael

[Eyal Itkin]

Hi,

The suggestion to add a new ld flag (--static-lib) and use it to localize symbols as is done in --shared, still leaves two important points unaddressed:

1. If internal function symbols (turned local) will keep their relocations, said symbols will fail to be stripped, and the binary will "leak" all function names. I am aware this also happens when -ffunction-sections is used (which is a different topic) yet developers can choose not to use this flag. Shouldn't there also be an option for closed-source projects to maximize the stripping of their binaries?

2. If no new e_type value will be added to the standard, what will be the basis of convincing the maintainers of ld to extend the linker to support a new format/behavior? I fail to understand what is the industry method of declaring this new format for static libraries which will then be followed by implementation by all toolchains.

I will also add that having a different e_type than ET_REL also means that the linker will be able to differentiate between the two. For instance, this can allow for dedup of static libraries at command line if pkg-config causes the same static library to be passed more than once (as often happens, and is cited in my paper). This once again brings us to static libraries being first class citizens (like shared libraries) which allows them the same dedup/stripping features as exist for shared libraries.

Thanks again,

Eyal

--

You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/generic-abi/75BE2C28-55B0-4EFC-AB85-744E0C265105%40espindo.la.

[Ali Bahrami]

Adding a new object type to the generic ABI faces a high hurdle,
because it will impose costs on anyone maintaining an ELF toolchain,
as well adding complexity to a standard that benefits greatly from being
relatively simple.

It's telling that in 40+ year run, ELF has only needed 4 types, REL, DYN,
EXEC, and CORE. There's nothing new about static linking, so it seems
surprising that a new type to cater to it would be needed.

I delayed commenting until I had a chance to read the paper. Having
read it, I agree with the others that this really still sounds like something
an ET_REL can handle, and agree particularly with Rafael's comment:

On 9/27/25 12:05 PM, Rafael Ávila de Espíndola wrote:
> Having said that, I think you make a good case for a new linker command
> line (--static-lib?), not a new elf file type.

If I understand this project correctly, your goal is to produce a
relocatable-like object that is self contained, like a shared object
would be, and which only exports the global symbols that make
up an interface. The idea is that you could link to this, much as
one does to a shared library, except that the bits will be copied
into the final object.

One big thing that distinguishes an executable or shared
object from a relocatable object, is that EXEC/DYN go through
a process of finalization, where things like symbol resolution
are done to turn them into complete objects. I believe that your
paper is describing the process of finalizing an ET_REL.

You can definitely do that. Solaris kernel modules are an example.
Our kmods are ET_REL, but are distinguished from regular ET_REL
by going through finalization. I can imagine optionally doing this for
"regular" relocatable objects. In reading your paper, I have the impression
that this is what you've already done, and I wonder what added benefit
comes from giving them this different type. Is there a reason that the
result needs to be identified specially as a static library, rather than as
just another relocatable object? Being able to make them ET_REL
would lower the number of changes needed greatly, for yourselves
as well as the rest of us.

- Ali

[Ali Bahrami]

On 9/27/25 1:29 PM, Eyal Itkin wrote:
> The suggestion to add a new ld flag (--static-lib) and use it to
> localize symbols as is done in --shared, still leaves two important
> points unaddressed:
> 1. If internal function symbols (turned local) will keep their
> relocations, said symbols will fail to be stripped, and the binary will
> "leak" all function names. I am aware this also happens when -ffunction-
> sections is used (which is a different topic) yet developers can choose
> not to use this flag. Shouldn't there also be an option for closed-
> source projects to maximize the stripping of their binaries?

Not as a primary motivation, no. Not because I care about hiding IP
one way or the other, but because I don't think that sort of obfuscation
is a very effective barrier to a determined disassembler.

That's not to say that you're stuck keeping those symbols
(as local) though. It is also possible to rewrite such
relocations to be section relative (use a section symbol), rather
than the original symbol. That achieves some obfuscation,
which might make someone trying to hide IP happy, but which
will probably making someone using a symbolic debugger to fix
it unhappy. That sort of rewriting is done when local symbols
are dropped, usually as the result of a special option that
causes it, or the use of a mapfile/linker-script.

I'm not clear on what special problem is caused by groups
(-ffunction-sections). I'd expect group processing to be one
of the things done by the finalization step mentioned previously.

> 2. If no new e_type value will be added to the standard, what will be
> the basis of convincing the maintainers of ld to extend the linker to
> support a new format/behavior? I fail to understand what is the industry
> method of declaring this new format for static libraries which will then
> be followed by implementation by all toolchains.

What added support is needed?

Clearly you need to know it when you create the thing,
so a creation time option is needed.

Afterwards though, it's just another .o, albeit one
that was built unusually. Most of what's been said here
is based on the assumption that there is nothing special
for the link-editor to do at that point. Not true?

>
> I will also add that having a different e_type than ET_REL also means
> that the linker will be able to differentiate between the two. For
> instance, this can allow for dedup of static libraries at command line
> if pkg-config causes the same static library to be passed more than once
> (as often happens, and is cited in my paper). This once again brings us
> to static libraries being first class citizens (like shared libraries)
> which allows them the same dedup/stripping features as exist for shared
> libraries.

By itself, that's not a strong reason to invent an entirely
new object type, which all ELF toolchains will be forced to
understand. If it really was seen to be a problem in practice,
there are probably smaller solutions.

- Ali

[connor horman]

One thing that a separate type can have that just ET_REL can't have (which generalizes the deduplication issue) is that linkers aren't going to treat two ET_REL files on the command line differently. However, static libraries ought to be treated more like .a files and ET_DYN objects. Notably, I'd expect any elf-based static library to be affected by `--as-needed` options like shared objects are. Similarly, I would not expect linkers to apply --as-needed to normal object files. Deduplication is indeed another issue. Many tools will happily add a static library to a command line because it is benign to do so if multiple copies end up (as long as you don't set --whole-archive). If static libraries are converted to an ELF-based solution, I would 100% expect them to be suitable for deduplication so that they retain this property. Frankly, I don't think ET_REL would be suitable as a library format for that reason - adding it to the link line is not idempotent.

--
You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/generic-abi/8ec96983-bed6-47a0-93c4-32f8e0e16eaa%40emvision.com.

[Rafael Ávila de Espíndola]

Hi,

A nice thing about elf is that a producer can decide how granular sections are. If you are ok with degrading the final result a bit, you could have --static-lib --combine-secs. It would instruct the linker to combine sections and remove relocations when possible.


The second point is better, as handling duplicates is something different about static libraries.

The question is if it is worth it. If you add the above options to any linker, you can stop shipping .a files tomorrow. With a new file type, you will need to wait for all customers to upgrade their linkers.

Thanks,
Rafael

[Rafael Ávila de Espíndola]

Ali Bahrami <ali_e...@emvision.com> writes:

>> I will also add that having a different e_type than ET_REL also means
>> that the linker will be able to differentiate between the two. For
>> instance, this can allow for dedup of static libraries at command line
>> if pkg-config causes the same static library to be passed more than once
>> (as often happens, and is cited in my paper). This once again brings us
>> to static libraries being first class citizens (like shared libraries)
>> which allows them the same dedup/stripping features as exist for shared
>> libraries.
> By itself, that's not a strong reason to invent an entirely
> new object type, which all ELF toolchains will be forced to
> understand. If it really was seen to be a problem in practice,
> there are probably smaller solutions.

I just realized that probably the simplest solution is to have the new
--static-lib option produce a .a file with a single .o file in it.

Cheers,
Rafael

[Eyal Itkin]

The notion of a phased deployment by shipping both a .a that includes a single ET_REL .o file and an .sbo file, later to deprecate the former, is indeed mentioned in my proposal.

One point that I'm still unsure about is the effort/timeline notion that was suggested in the thread. No change will be immediate - Any change that hopes to get wide adoption by users will need to wait for support that will be added to ld (if required) and that will gradually propagate to a release in an LTS distro of one OS or another, to be picked up by projects. Projects nowadays often don't use a Makefile, but some other technology on top of it, such as Bazel, CMake or Meson. These technologies will need to support whatever proposal will be decided (objcopy commands, ld linker commands, etc.) and offer it to users through their API. For compatibility reasons, every solution aside from a .a archive that wraps something, will also impact packaging, pkg-config definitions, and build API if a phased approach is being used or a project wishes to switch to the new offering.

These steps will be needed for any replacement of the current static .a archives, so none of them will be immediate for the average user. And while arcane ld flags or objcopy flags might be available today, the average user that uses Bazel/CMake/Meson will never have access to these tools out of the box, meaning that most projects will either never use them, or will have to integrate scripts for these tools on their own in their build environment, multiplied by all projects out there.

I do understand that adding an enum value will cause work for all toolchains. The addition of the enum option itself for binutils was implemented by me as part of the POC, and it is a very minor commit if one wishes to only be aware of the format without yet supporting it.

However, doesn't any decision of this forum (put aside adding new e_machine values) is a decision that impacts all toolchains? And that by definition adds/extends something that wasn't defined (or at least fully defined) in the 40+ years of this standard? Do we really want to fix only some of the issues projects complain about when using static archives (such as the 3 years  old rust bug for the standard library), and leave some of them open, including keeping the hack of using a .a archive to wrap whatever we replaced under the hood?

I never anticipated that a mainstream project would be able to use the new proposed format before Ubuntu 28.04. And I have the time and commitment to help relevant projects to prepare and integrate the required updates. But this will only be relevant if we start at some point. If, instead, we keep the notion that the last 40+ years were good enough, then the next 40+ years will look exactly the same and no progress will ever be made.

--
You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/generic-abi/87plba4xbn.fsf%40espindo.la.

[Eyal Itkin]

Going over the thread and trying to summarize the points that were raised, it seems that there are roughly the following four main options.

Option #1 - Do nothing. No substantial claim was made to showcase the existence of a gap that requires addressing, or that is worth being addressed.

Option #2 - This is a tooling issue. objcopy and ld might need to offer improved features so to handle symbol scoping issues (scoping, not stripping). Please note that objcopy might localize symbols yet won't finalize the ET_REL, hence relocation will be kept, internal symbols will be kept, and no stripping will be available. I doubt that use of objcopy flags will be widely adopted by the average project, making this more of a niche solution. If a change to ld will be introduced, it would be in the form of finalizing relocations, which pretty much brings us to option #3.

Option #3 - The linker should support --static-library flag. There is indeed a strong case for a new static library format, replacing the existing .a archive. The new linker flag will create an ET_REL object as this falls within the scope of the existing e_type, and the process will support some sort of finalization to relocations, even if probably lesser than offered for the ET_DYN case. Still, there is no strong case for the linker to treat the static libraries differently than plain object-code files, unlike the way shared libraries are handled. This will prevent the ability to dedup static libraries from the command line, or the ability to have an "--as-needed" equivalent. I will also point out that "--ffunction-sections" alongside "--gc-sections" is out of the question for closed-source projects, as by definition the sections names are the function names, which cancels some of the benefits we wanted to achieve in the finalization step. The leading case is to keep on bundling this ET_REL file within a .a archive, so to maintain compatibility. It is unclear what are the guidelines by which ld will implement said support, as it would still require some formal recognition and standardization.

Option #4 - Add a new e_type (ET_STAT) and treat static libraries as first-class citizens. There is a strong case to allow the linker (and other tools) to differentiate between plain ET_REL files and static libraries, hence a new e_type is needed. In addition, option #3 already requires ld to do all the heavy lifting, hence the only delta is the added support for dedup on command line and "--as-needed" equivalents (which is pretty minor). There will most probably be additional cases in the future (or that we haven't articulated yet) that require distinction between libraries and plain object-code files, and this provides the infrastructure for doing so. Having a new e_type will help communicate that both Shared Libraries and Static Libraries are well defined formats which are defined and supported by the ELF standard. This will promote the adoption of the new format throughout toolchain and compilation wrappers (Bazel/CMake/Meson) helping to deprecate the existing static archives. Format will be shipped on it's own (.sbo file, without .a file) and could use a phased approach as suggested in the paper.

Regards,

Eyal

[Florian Weimer]

* Ali Bahrami:

> It's telling that in 40+ year run, ELF has only needed 4 types, REL, DYN,
> EXEC, and CORE. There's nothing new about static linking, so it seems
> surprising that a new type to cater to it would be needed.

It might be that new ELF types are hard to deploy. For example,
seperated debuginfo would benefit from a new type to reduce confusion,
but existing consumers apparently check that the ELF type of separated
debuginfo matches that of the executable file.

Thanks,
Florian

[Ali Bahrami]

I'm sure that plays a role. Another is that such a type
in the core gABI would have to suit the needs of everyone,
but we don't have that sort of consensus for separate debug.
For instance, the Solaris OSABI has ET_SUNW_ANCILLARY and GNU
have separated debuginfo. They're very different, and there's
not much interest on either side in merging or replacing them.

ET_SUNW_ANCILLARY is useful --- it allows tools like 'file',
and debuggers, to work cheaply and without guesswork. But of
course the bar is lower for an OSABI addition. I can't imagine
that it would have been accepted by the gABI. In the case of
debuginfo, it might have been easier if it had started with
an ET_GNU_DEBUGINFO type, but retrofitting it now would
certainly be troublesome.

- Ali

[Eyal Itkin]

* Rafael Ávila de Espíndola

Sep 28, 2025, 2:43:16 PM (4 days ago) 

>> I just realized that probable the simplest solution is to have the new

>> --static-lib option produce a .a file with a single .o file in it.

A lot of today's integration problems are a combination of projects using "--whole-archive" and the lack of "--as-needed" for static libraries when doing so. Given that only a single .o file will be placed within the archive, "--whole-archive" won't be needed in these cases, and the native behavior is already aligned with "--as-needed", so this indeed addresses both of these issues.

I won't say that keeping the use of a .a archive of a single .o file is the most elegant solution, yet it ticks all the boxes, and that is good enough.

What is needed now in order to formally define this "--static-lib" linking option? I would be more than happy to assist where possible, whether it is in discussions, paperwork or the coding itself with GNU's ld or other toolchains.

Thanks again,

Eyal

[Rafael Ávila de Espíndola]

Given that from the client point of view it is just another .a, I think
that all that is needed is sending patches to your linker of choice
implementing --static-lib.

Cheers,
Rafael

[Eyal Itkin]

I'm still new to this process so please excuse me for asking, but are you telling me that there is no need anywhere to define the new "standard" for static libraries? Or that your meaning is that what we discussed in this thread is good enough?

I'm asking because I'm trying to wrap my head around what will be the motivation of the maintainers of GNU ld to discuss patches that I'll prepare instead of telling me that this sounds like an ELF standard issue and that a new e_type is needed? And even if my patches will be accepted, what blocks other toolchains from deciding they don't think they should support a "--static-lib" flag? Or maybe they want to support it using a completely different binary format?

After all, while there is no change at the client side, there is the change at the project side, as this change should propagate to: 1) The linker, 2) The build infrastructure that will pass this flag to the linker. Hence, the "news" about this proposed format should still travel around the community so that it would help with the adoption of it.

Regards,

Eyal

On Friday, October 3, 2025 at 11:16:08 AM UTC+3 Rafael Ávila de Espíndola wrote:

Eyal Itkin writes:

[Ian Lance Taylor]

On Fri, Oct 3, 2025 at 1:33 AM Eyal Itkin <eyal....@gmail.com> wrote:

I'm still new to this process so please excuse me for asking, but are you telling me that there is no need anywhere to define the new "standard" for static libraries? Or that your meaning is that what we discussed in this thread is good enough?

I'm asking because I'm trying to wrap my head around what will be the motivation of the maintainers of GNU ld to discuss patches that I'll prepare instead of telling me that this sounds like an ELF standard issue and that a new e_type is needed? And even if my patches will be accepted, what blocks other toolchains from deciding they don't think they should support a "--static-lib" flag? Or maybe they want to support it using a completely different binary format?

After all, while there is no change at the client side, there is the change at the project side, as this change should propagate to: 1) The linker, 2) The build infrastructure that will pass this flag to the linker. Hence, the "news" about this proposed format should still travel around the community so that it would help with the adoption of it.

If the ELF standard adds a new e_type, then the linker still has to change to support it. And a bunch of other tools also have to change to support it. Making a change to the ELF standard does not somehow cause linkers, and other tools, to also change. Conversely, many tools extend the ELF standard in various ways. Change does not start at the ELF standard and flow outward to other tools.

If a tool picks up a useful feature then other tools will adopt it. So if you want to make a change, start by changing the tools.

Ian

[Ali Bahrami]

On 10/2/25 2:40 PM, Eyal Itkin wrote:
> I won't say that keeping the use of a .a archive of a single .o file is
> the most elegant solution, yet it ticks all the boxes, and that is good
> enough.

I think it ticks many boxes that you're not even
thinking about, because sticking to these existing
mechanisms immediately expands your reach to just about
every ELF system, and not just the ones you've directly
set out to care about.

For example, I'm sure you're not concerned about Solaris,
but even so, from the sound of it, all I would need to use
this there would be to upgrade to a newer gcc/binutils.
Now adding this new option to the native Solaris link-editor
is definitely not free, but since it doesn't involve a format
change, and can probably reuse quite a lot of existing code,
it's also not particularly hard. Whether I'll actually do that
or not really just comes down to demand from my users. In the
meantime, gld is enough to let me play, and the price is right.
The low barrier to entry does qualify as elegant, even if
the idea of having one object in an archive might not.

On 10/3/25 2:33 AM, Eyal Itkin wrote:
> I'm still new to this process so please excuse me for asking, but are
> you telling me that there is no need anywhere to define the new
> "standard" for static libraries? Or that your meaning is that what we
> discussed in this thread is good enough?
>
> I'm asking because I'm trying to wrap my head around what will be the
> motivation of the maintainers of GNU ld to discuss patches that I'll
> prepare instead of telling me that this sounds like an ELF standard
> issue and that a new e_type is needed? And even if my patches will be
> accepted, what blocks other toolchains from deciding they don't think
> they should support a "--static-lib" flag? Or maybe they want to support
> it using a completely different binary format?
>
> After all, while there is no change at the client side, there is the
> change at the project side, as this change should propagate to: 1) The
> linker, 2) The build infrastructure that will pass this flag to the
> linker. Hence, the "news" about this proposed format should still travel
> around the community so that it would help with the adoption of it.

I don't think putting a single object into an archive is
a new idea, nor one that needs standardizing. Nor are the
sort of finalizing steps you'll be applying to your generated
.o. Since all of that is built out of existing pieces, the GNU
ld folks should have no standards concerns about it. They'll
probably have lots of other concerns, as with any project.

Finally, as Ian said, you don't standardize something like this
and just expect the users to follow. What you do is roll it out,
start using it, refine it based on experience, and win users as
they see value. Even if you ultimately did need a standards change,
this would be the necessary first step, because there's nothing worse
than standardizing something that turns out to need tweaking later.
New gABI additions need to be already proven somehow.

So yes, I think you can just carry on from here.

----
Nothing that follows is gABI related, and are just thoughts
that have occurred to me as this has gone by.

I wonder about calling these "static libraries", because
that expression has long been used to describe ordinary
archives containing non-PIC objects, intended to be linked
into executables. What you're doing is something else:
You're producing a single relocatable object that's been
finalized so that it provides an entire unit of things.
That unit is most likely a "library", but it could be
any self contained collection of objects.

I wonder if there's a better name that isn't ridiculously
verbose (e.g. --finalize), that would be used with -r,
that better captures the essential thing being done.

Related to that, I'm not sure that you should have the
link-editor produce the archive, containing that object.
I'd vote for producing the object, and letting the user
stuff it into an archive if they need those additional
features. This is more flexible:

- I don't have to use an archive, if my project
doesn't need that, I can just link to the .o.

- I can put it in another archive with other, unrelated, objects,
if that model better suites what I'm building. You could
essentially have a single archive, containing multiple
finalized "library" objects, any of which will only be
extracted on demand, like any archive.

Which is another reason not to call it a "library", and instead
try to find a name that captures the essential operation being
done to those objects.

Good luck!

- Ali

[Cary Coutant]

The main argument I see here for the ET_STAT file type is that you want the visibility flags (STV_) to work the way they do in a shared library; in a relocatable object, they are ignored for linking purposes, and simply passed through to the output file (with some resolution rules governing conflicting visibilities for a symbol). I don't think that you've yet made your case for that, though.

If we're going to reimagine static (relocatable) libraries, let's step back. Here's a list of features I'd consider important for a relocatable library to support:

1. Granularity. This is really the key feature. A client of a runtime library shouldn't have to link in every module from that library, if it uses only a small portion of it. Linking the whole library is wasteful (in most cases). It makes sense for shared libraries because they're shared — we load one copy of the whole library, but share that one copy among all applications that bind to it. (I'll note here that the AIX linker, as I understand it, does, in fact, link in all modules from an archive library, then does dead-code elimination to throw away the modules that weren't needed. Most other linkers just search the library and pull in modules to satisfy references. It's a subtle difference that usually has the same result.)

2. Random access. With granularity, we want the ability to randomly access the modules in the library. In the early days of computing, libraries had to be topologically sorted so that each module preceded those it depended on. Early Unix had an lorder script that would reorder the modules in an archive library; later versions added the library symbol table as a hidden member of the archive.

3. Namespace isolation. The solution we have now is rather limited (dare I say "weak"?). Most any library is going to have internal entry points it doesn't want to expose to the client. The system libraries do this by using the reserved namespace (e.g., symbols starting with a double underscore) for those functions. Third-party libraries need to invent their own namespace that's unlikely to clash with the client application's namespace.

4. Full relocatability. For library code that's not going to be shared, we want to keep it fully relocatable. It should benefit from dead-code elimination, relaxation, and profile-based optimizations like code re-ordering.

5. Packaging. It should be packaged as a single file, and identifiable as a library rather than a plain object file. The archive (.a) format serves this purpose, as would your proposed ET_STAT file type. It could also be a different kind of ELF file with extensions that preserve the notion of individual compilation units and the granularity of the archive library format.

6. (Not really relevant, but I'll throw this in...) Hybrid relocatable/shared support. Some third parties might want to package the library as a shared library with some relocatable components that should be statically linked into the client application. (Windows uses this approach to provide the linkage stubs for DLLs, rather than having the linker build a PLT. I've run across potential uses for something like this over the years—I just don't remember them offhand.)

On granularity, you've listed that as a problem, but I disagree. One problem you mentioned was static constructors (which is a subset of SHT_INIT/INIT_ARRAY code) that doesn't get pulled in from a library unless some other entry point in the same module is pulled in. That's actually a feature, but yes, there are cases where a library developer wants to force some initialization to get linked. Workarounds for that are the -u option and the sledgehammer --whole-archive option, both of which have the disadvantage of burdening the client with out-of-band knowledge on how to link against the library (e.g., with a "package config" file or similar). You could also add a symbolic dependency from every other object in the library back to the one you want forced in. That would make sure the initializer object gets linked if any other object is linked (better approximating the "as-needed" option for shared libraries).

I'd suggest a better approach is to keep that knowledge in the source code. GCC already supports __attribute__ ((used)) to mark a function that must be present even if it's not referenced. Unfortunately, that attribute doesn't propagate to the object file, so the function is still subject to dead-code elimination. If we were to add an SHF_REQUIRED section flag, GCC could mark the section containing that function as required, keeping it from being removed via dead-code elimination. That still wouldn't force the module containing that section to be linked from an archive, but we could add a feature to the archive format that marks the object as required — one way to do that would be to add a ".FORCE" symbol to the library symbol table. (Perhaps it's too aggressive to apply this to all symbols marked "used", so an alternative would be to add a new "required" attribute instead of overloading "used".)

On namespace isolation, you've suggested using the symbol visibility attributes as a way of restricting the visibility of the internal entry linkages within the library, to mirror the way it works for shared libraries. Others have suggested using linker options to internalize symbols during an "ld -r" link, but (as you noted) that's a burden on the library build process. If we keep the idea of a relocatable library as a collection of individual compilation units (which I'd prefer), an alternative idea might be to add a new binding type, STB_LIBLOCAL, which acts like STB_GLOBAL for references from within the same library, but would not resolve references from outside objects.

On full relocatability, you seem to be favoring the idea of doing as much binding as possible, resolving relocations wherever possible. While you could resolve pc-relative relocations within a section, even that's not possible if you expect the linker to perform certain kinds of relaxations. You could go ahead and "segment" the code like you would when building a shared library, but then those segments would just be statically copied into the output file and you'd have to have a separate PLT and GOT to handle the linkages in and out of the library code.

You've also mentioned some forms of "code obfuscation," which I haven't listed among my desired features. If you take the "ld -r" approach, you can localize all the internal symbols and remove local symbols from the link, converting references to those symbols into section-symbol-relative references. That's not quite so practical if you keep the modules of a library separate, as in an archive library—you'd have to resort to using some sort of tool to obfuscate the global-but-library-local symbols. To have both granularity and internal symbol elimination, we'd want to come up with a package that allows local inter-module references while keeping the modules otherwise independent.

While I'm sure your proposal for ET_STAT suits your needs, I don't think it's the direction I'd like to see relocatable libraries move towards in general, and it seems like you've been offered several suggestions for accomplishing your goals with existing tools.

-cary

[Cary Coutant]

Related to that, I'm not sure that you should have the
link-editor produce the archive, containing that object.
I'd vote for producing the object, and letting the user
stuff it into an archive if they need those additional
features. This is more flexible:

     - I don't have to use an archive, if my project
       doesn't need that, I can just link to the .o.

     - I can put it in another archive with other, unrelated, objects,
       if that model better suites what I'm building. You could
       essentially have a single archive, containing multiple
       finalized "library" objects, any of which will only be
       extracted on demand, like any archive.

I've also seen people simply rename a .o file to .a. That lets you use the -l linker option, but the linker (usually) will recognize the file as a single object and link it in unconditionally.

-cary

[Eyal Itkin]

Update: Following the recommendations from members of this thread, I have submitted an initial draft for review on GNU ld's mailing list - https://sourceware.org/pipermail/binutils/2025-October/144795.html. However, given that some members of this thread have continued the architecture-level discussion about the proposed feature on GNU binutils's thread, I thought it will be best to consolidate back both threads so to improve clarity and readability by having the architecture-level discussion hosted in a single place. This is especially important given some glitches in binutils's archiving which for some reason corrupted the tracking of the original thread by dropping some of the emails from it. Hopefully, once the architecture is clarified, the discussion for each tool (GNU ld, Clang, etc) will be focused on design and coding level issues (files, functions, naming, testing) instead of the root architecture of the feature.

As a short recap, the top goal we are trying to achieve is to allow developers of 3rd party static libraries to mask out internal implementation details (effectively creating an abstraction layer) so that users of said libraries won't face integration issues such as symbol conflicts with internal library symbols. In the whitepaper this is referred to as the "scoping" issue, and it was demonstrated to encompass all non-static functions within the static library, extending to also include static library A swallowing static library B.

The origins of the issue can be easily visualized by inspecting the graph that is traversed by the linker-editor during the linking process. Today, a static library is simply a bag of .o files, which have internal edges between them (internal API) and external edges from them to other nodes in the graph (external API). Semantically, the set of vertices in the graph (one per .o file) can be surrounded by a large virtual vertex that simulates the static library. During the linking process the linker-editor will traverse an edge (symbol that is being resolved) into one of the library's vertices, and will continue on traversing the edges until no more edges (internal or external) remain to be traversed (all symbols were satisfied).

Today, the linker de-facto *ignores* the virtual vertex that marks the vertices of the static library, and treats all edges as equal, whether these are API functions of the library (marked as STB_GLOBAL), or internal non-static functions within the library (which are also marked as STB_GLOBAL). As such, the linker can face a "symbol conflict" between both types of edges. Any attempt to use objcopy will be futile, as it will only be able to remove a given edge in the graph (localizing a symbol) yet this will block the linker from resolving this needed symbol, hence won't achieve the desired goal.

Under the current behavior, the two main proposals for resolving this issue, as raised in this thread, are the following:

Option #1 - Bundle together all .o files of the static library so that the linker will treat them as a single vertex in the graph. This will allow us to localize the symbols on the internal edges, effectively treating them just like static functions within a single compilation unit.

Option #2 - Create a new marking so that external edges (STB_GLOBAL) and internal edges (STB_LIBLOCAL) will be marked differently, and educate the linker about this new behavior.

For the sake of clarity I will refer to the new attribute of option #2 as "STB_LIBGLOBAL" instead of "STB_LIBLOCAL" given that it is "global" in the context of the library and "local" in all other aspects.

Before diving into the different options, it is important to emphasize the following points:

* The proposed solution is optional, and there is no intent on forcing users to use it instead of the existing archive of plain .a files.

* The proposed solution does not intend to be a silver bullet that will solve all cases. I am of the belief that perfect is the enemy of good, and that any solution that will be chosen will have some tradeoffs.

* The goal is to provide developers yet another tool so to enrich their toolbox, and empower them to decide what fits their needs.

* The proposed solution acknowledges that different users have different requirements and priorities in mind. While some users prioritize binary granularity during linking, some users do not have this requirement (as is evident by the existing of the "--whole-archive" flag or the https://github.com/tux3/armerge utility).

Option #1 - Bundling the .o files - This is the option I chose to implement:

Given the linker's existing "ld -r" support for creating a large relocatable that amalgamates all input .o files, the proposed design is straightforward.

Step A (will be patch #1) - Add support for symbol visibility (fvisibility=hidden, --version-script=version.map). This will reuse existing CLI flags and code as is already implemented for shared objects, and will require less than 50 lines of code. While this might be a "duplicate effort" given that objcopy already supports similar features, I do not believe this to be the case. If we anyway instruct users to pass through the linker when building their static library, adding another step of passing also through objcopy seems wasteful. In addition, objcopy does not support version-script syntax (only a flat file syntax), which will forces SDK developers to maintain a duplicate list of their symbols, one per library type. Please also note that the version-script syntax is the only one that will support the case of library A swallowing library B, as the original symbol visibility in library B should be updated also for the public API functions (from global to local).

Step B (will be patch #2) - Add CLI flag for finalizing local symbol relocations (off by default). This feature is already a built-in part of the "-shared" flag, yet will be added as a separate and optional flag for the relocatable case. This implementation was shown to reuse logic from the shared library case, and can also fit in less than 50 lines of code per architecture. Given the implementation for shared libraries, I am of the opinion that this is the semantically correct thing to do, even if it is only a best effort approach that won't cover all relocations (such as cross-section relocations). Initial checks on x64 using DPDK's librte_eal.a library shows 96% success rate for finalizing and eliminating internal symbols. Initial inspection of the aarch64 case shows promising prospects as well. On top of the semantic value of it, this finalization will allow stripping of (most) internal symbols, as is done for shared objects, and will address the "exposure issue" as listed in the technical paper.

An important observation is that while step B requires step A, they will be implemented independently. This allows users to decide if they only wish to use step A in their project, use both  of them, or even prefer to skip them all entirely (as it also supported today for creating a shared object without symbol visibility).

The created relocatable (.o file) will be added to a .a archive using the ar utility (just like before), potentially allowing SDK developers to bundle all of their "libraries" inside a single .a file instead of using the existing " --start-group" approach for some needed scenarios.

Option #2 - Introducing a new STB_LIBGLOBAL attribute while keeping the bag of .o files. While this feature has promising granularity aspects, I'm not entirely certain how will it be supported in practice.

One option is that the compiler produced .o files will already be marked accordingly. This forces the user to mark their functions at the code level accordingly, or to add a compilation flag to differentiate between source files that are compiled for libraries, and source files that are compiled for full blown executables. The latter option is potentially wasteful as it will require two different outputs for the same source file, which one time will be part of an executable (runtime utility) and one time part of a library. If marked at the compilation level, this will leave the case of library A swallowing library B as an open gap as the .o files were already marked differently than what we require from them (global instead of libglobal). 

Another option, is that the compilation will be kept the same, and some additional processing step will be done, either through ld or more likely through objcopy. Using objcopy will once again suffer from the lack of version-script support, and will introduce a complication. Today many software projects compile the sources files into .o files, and then re-use them for both static and shared libraries, thus significantly reducing compilation time and disk space. Any attempt to create a "processed" version of these .o files prior to adding them to the .a archive will require additional compute and special attention so not to mix the two.

On a semantic level, the second option will also imply that creating a static library required a compiler, objcopy and ar, yet not the linker. On top of being far from elegant, this will also imply that objcopy is now a material part in compiling a standard static library, while today it is mostly referred to by users as a swiss-army-knife tool for special occasions (many times in embedded toolchains).

One last important part is that even after introducing STB_LIBGLOBAL, the linker should be extended to support it at the "client" side. This means that any hope of using this new approach will require full propagation to all users before the feature can be enabled. This is in contrast to the first approach (using "ld -r") that only requires a change on the developer side, without any new requirements on the client side (the integrator of the library).

Will appreciate any notes or feedback that you might have on the above. In the meantime (assuming no hard rejects) I will continue on with the implementation of option #1 at GNU ld's level, and will continue the review process with the project's maintainers.

Thanks again for your assistance and effort,

Eyal.

[connor horman]

I thought of some benefit that would differentiate `ET_STAT` from ET_REL further than simply denoting the kind of relocatable object, though it would require substantially more toolchain work - it could be allowed that ET_STAT libraries can contain a "Partial Program Header" in which case, the resulting library would be placed at a single consistent base address within the module. This allows two transformations:
1. All relative relocations against a non-interposeable definition within the library can be resolved, even cross section, and

2. All absolute relocations against a non-interposeable definition within the library can be replaced with an R_*_RELATIVE relocation (or R_*_IRELATIVE, in the case of an ifunc symbol where that is supported).

However, it would still not allow reducing GOT relocations (except as an optimization where the relocation is against a non-interposeable definition) or TLS relocations.

This would require the link editor to be aware of this behaviour, though (including linker script changes) and not merely acknowledge ET_STAT as a sibling for ET_REL (with ET_DYN-like CLI semantics).

--
You received this message because you are subscribed to the Google Groups "Generic System V Application Binary Interface" group.
To unsubscribe from this group and stop receiving emails from it, send an email to generic-abi...@googlegroups.com.

To view this discussion visit https://groups.google.com/d/msgid/generic-abi/3a125060-3fe1-4103-a9a4-1296484e98fen%40googlegroups.com.