> Can you elaborate on what the use cases are for comdat groups in ELF if we
> aren't putting metadata (debug info, .eh_frame, asan global metadata, etc)
> in the group? I'm aware of GCC's destructor alias optimization involving D5,
> but that doesn't seem like an important enough use case to guide the design
> and usage of ELF comdat groups.
It was always intended that a whole collection of sections associated
with the function would be in the comdat group -- it was essentially
supposed to be an "object file within an object file". The idea was
that associated code, data, unwind, debug, etc., sections would all be
in the one group, and could be kept or discarded as a whole group.
(You talk about "metadata", but it's more than just metadata.) This
was an improvement over prior template instantiation schemes, which
involved either (a) a post-link step to identify missing
instantiations, compile them, then relink, (b) using weak symbols to
suppress the multiple-definition complaints, but otherwise living with
duplicate code (possibly relying on garbage collection to clean things
up), or (c) using "link-once" sections but no grouping.
Unfortunately, due to the nature of DWARF debug information, and the
way most compilers generated it, it wasn't practical to put the debug
info in the group -- at least in GCC and other compilers I'm familiar
with -- and we had to live with the duplication (which could get quite
unwieldy in large projects, sadly). I think GCC has always been a bit
primitive, compared to some other vendor compilers, with regards to
including other associated sections in the comdat groups, and I
suspect this is related to the simple conversion from the old
"link-once" sections that GCC had used prior to comdat groups.
> I'm not sure what you mean about how they were intended to be used for
> template instantiation. Was the idea that all of an explicit
> 'std::vector<int>' instantiation would live in a single comdat group? That's
> definitely not the way things seem to operate today.
No -- all the sections for each individual function were meant to be
part of a single group, but each function gets its own group.
> It's possible that I am alone in imagining the rough equivalence between ELF
> comdat groups and COFF associative comdats, but it seemed pretty clear to me
> that ELF comdat groups were intended to be used for metadata the way that
> COFF associative comdat sections are.
I'm not familiar enough with PE-COFF to give you a good answer. We
definitely borrowed the idea (and the name) of "COMDAT" from
Microsoft, but we designed it specifically for the needs of template
instantiation rather than copying the exact specification.
> In fact, the more I've learned about the way things work on ELF, the more
> I've been surprised at how much duplicate metadata remains after the final
> link step. For example, both clang and gcc seem to construct monolithic
> .eh_frame and .debug_info sections full of information describing inline
> functions that may be discarded. Going forward, if DWARF producers wanted to
> try to eliminate this metadata from the final link, do you think we should
> use comdat groups or this new flag? I think whatever ELF uses to drop
> duplicate DWARF would also apply to our asan global metadata use case.
Yes, it's regrettable. I think some of the vendor compilers (e.g.,
Sun, HP, IBM) may do better, but I don't think anyone has yet solved
the DWARF issues. The DWARF spec might need some small enhancements to
make it practical, and I've had that on my list of ideas to work on
for years now.
>> Comdats make it possible for the linker to always select a paired foo
>> and bar. And it doesn't even have to read the sections of unselected
>> comdats.
>
> I'm not aware of any C++ compilers that put foo and bar into a comdat group.
> If you want to support inlining foo, you need to keep bar in its own comdat
> group. In fact, there are tons of ABI compatibility bugs around this,
> precisely because different TUs make different decisions about the
> readonly-ness of bar.
No, you don't need to put bar in its own comdat group, as long as you
always generate the out-of-line instance of foo along with it. You do
need to globalize the name of bar, so that it can be referenced from
any inlined code outside of the comdat group.
The requirement that each instance of a comdat group must define the
same set of global symbols is critical, and failure to follow that
rule has been the cause of many mysterious "undefined symbol" errors
and other bugs.
-cary