New linker

[Amnon]

What is the best place to read about the ongoing work on the new linker?
With the Go 1.15 release, how far along are we with the plans?
What has been achieved?
And what is still to come?

[Ian Lance Taylor]

The work on the new linker is close to complete and has been merged
into tip for the future 1.16 release (although 1.16 won't be released
until next February). The work is as described at
https://golang.org/s/better-linker.

Ian

[Amnon]

Thanks for the answer!

Thanks also for your blog post on linkers https://www.airs.com/blog/archives/38
I know it is from 12 years ago. But I am behind on my reading, and it is taking 
me some time to catch up. And it still is an excellent introduction to what linkers do.

Yes Austin Clement's better-linker document is great. But it is nearly a year old.
So I was wondering if there were any changes to plan, surprises or new insights.

Austin's doc refers to your idea of a new object format for the 21st century. 
Has anyone developed that?

As someone who has been writing linkers since 1988, (most famously the Gold linker), and who has a fair 
number of commits on the dev.linker branch,  do you have any insights into the new linker project
which would be interesting to share with the world?

The new linker is probably the biggest change in 1.15. But there is surprisingly little information about it.
It is almost invisible - which I suppose is a tribute to its success.
But are there any talks about the new linker which are worth watching?

- Amnon

[Amnon]

As an aside, for those who do not know, Ian wrote the Gold linker, which sped up linking by a factor of 100.
https://lwn.net/Articles/276782/ provides an index to his 20 part blog post on linkers.

[Than McIntosh]

To add to what Ian mentioned:

Many of the measures outlined in https://golang.org/s/better-linker have been implemented at this point.  It was a multi-person effort, several members of the Go compiler/runtime team were involved (Jeremy, Cherry, Austin, and myself).

The new indexed object file format and the improved deadcode implementation went into the tree in time to be included in Go 1.15. 

The additional changes in Go 1.16 include better streaming of data and relocations from object file to linker output, a rewrite of the pclntab code, support for content-addressable symbols, and more complete cleanup of the old legacy linker data structures (e.g. removal of sym.Symbol).

Some things mentioned in the doc that didn't get implemented include moving to DWARF 5, and moving DWARF type generation from the linker to the compiler.

HTH.

Cheers, Than

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXEK_OWZxKOUieHBdoH4R0eyddu_cMf_p_udWSKgZsHdA%40mail.gmail.com.

[Ian Lance Taylor]

On Sat, Aug 15, 2020 at 11:49 PM Amnon <amn...@gmail.com> wrote:
>
> Austin's doc refers to your idea of a new object format for the 21st century.
> Has anyone developed that?

No, that is still just a thought.

> As someone who has been writing linkers since 1988, (most famously the Gold linker), and who has a fair
> number of commits on the dev.linker branch, do you have any insights into the new linker project
> which would be interesting to share with the world?

Thanks for the praise. Sorry to say that I don't have any particular
insights. I think the new linker has been great work, but I
personally was only peripherally involved. I read the design doc, and
that's about it.

Ian

[Amnon]

Ian, Than - thanks for the info and the hard work.
Looking forward to putting the new linker through its paces!

[Misha Gusarov]

Hello Than,

Are there plans to implement remaining pieces (I'm interested
ing moving DWARF generation from linker to compiler) for 1.17, or is the
project considered done for now?

Best,
Misha.

To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CA%2BUr55EHRz5O3HvdP6xwOZyrHWKJA-Oc%2B96nMOCnB%3DmzABTYqA%40mail.gmail.com.

--
Misha

[Than McIntosh]

To be clear, a fair amount of DWARF generation is already done in the compiler (bulk of the line table, subprogram DIEs and inline DIEs, debug ranges, etc).

As far as I know, there are no near term plans (at least within the compiler+runtime team) to work on moving  other bits of DWARF generation to the compiler from the linker, but I don't think there would be any objection to someone else doing it.

Than