Re: [SCM] glibc maintenance branch, t/hooks, created. glibc-2.12-869-g56798c4

[Thomas Schwinge]

Hi!

On Mon, 23 Apr 2012 14:38:41 +0000, Samuel Thibault <samuel....@ens-lyon.org> wrote:
> The branch, t/hooks has been created
> at 56798c444bc584c118b69a3506c4050b34edc35f (commit)
>
> - Log -----------------------------------------------------------------
> commit 56798c444bc584c118b69a3506c4050b34edc35f
> Author: Samuel Thibault <samuel....@ens-lyon.org>
> Date: Sat Apr 21 21:03:12 2012 +0200
>
> Add link rules to sort hooks
>
> otherwise they are not properly recorded

What is this about, what kind of issue does it address? (I'm afraid I
don't know a lot about this hooks stuff.)

This branch is missing from tschwinge/Roger_Whittaker (perhaps on purpose
-- please in such cases feel free to add comments to .topmsg saying that
something's not ready yet), as is t/libpthread_depends, but the latter
one won't be needed until libpthread is ready for integration into glibc.


Grüße,
Thomas

[Thomas Schwinge]

Hi!

On Thu, 10 May 2012 11:17:01 +0800, I wrote:
> On Mon, 23 Apr 2012 14:38:41 +0000, Samuel Thibault <samuel....@ens-lyon.org> wrote:
> > The branch, t/hooks has been created

> This branch is missing from tschwinge/Roger_Whittaker (perhaps on purpose
> -- please in such cases feel free to add comments to .topmsg saying that
> something's not ready yet), as is t/libpthread_depends, but the latter
> one won't be needed until libpthread is ready for integration into glibc.

Inspecting the current content of tschwinge/Roger_Whittaker, it turns out
that these two branches in fact have been merged before, but there's no
trace of that visibile for .topdeps and I have no idea how that is
possible at all -- do you happen to remember what you've been doing on
2012-04-21 (commit 6a0800259dc141930c149b9472b4be3db6858c3f and
thereabouts). Anyway, I fixed this now.


Grüße,
Thomas

[Thomas Schwinge]

Hi!

It is possible that this happened when only
refs/top-bases/tschwinge/Roger_Whittaker was updated on Savannah, but not
tschwinge/Roger_Whittaker. This might have happened if the former was
pushed first and then the connection terminated, for example. (Though,
my recent push first updated tschwinge/Roger_Whittaker and then
refs/top-bases/tschwinge/Roger_Whittaker.)


Grüße,
Thomas

[Samuel Thibault]

Thomas Schwinge, le Thu 10 May 2012 11:17:01 +0800, a écrit :
> On Mon, 23 Apr 2012 14:38:41 +0000, Samuel Thibault <samuel....@ens-lyon.org> wrote:
> > The branch, t/hooks has been created

> > at 56798c444bc584c118b69a3506c4050b34edc35f (commit)
> >
> > - Log -----------------------------------------------------------------
> > commit 56798c444bc584c118b69a3506c4050b34edc35f
> > Author: Samuel Thibault <samuel....@ens-lyon.org>
> > Date: Sat Apr 21 21:03:12 2012 +0200
> >
> > Add link rules to sort hooks
> >
> > otherwise they are not properly recorded
>
> What is this about, what kind of issue does it address? (I'm afraid I
> don't know a lot about this hooks stuff.)

See the RUN_HOOK macro. It needs that all the symbols referenced by
text_set_element are folded between the __stop__ and __start__ symbols,
otherwise the for loop doesn't find them.

Samuel

[Thomas Schwinge]

Hi!

Thanks, that helps.

How did this work before, though? Due to __symbol_set_attribute
specifying weak linkage for the static case, I can see why we didn't get
undefined symbol errors when linking for the static case, but what about
the dynamic case? (And that'd mean the whole loop just was a no-op
before, right?)

Did you find this while working on a specific problem?


Also, what about data_set_element which is only used in hurd/dtable.c for
_hurd_fork_locks, which is manually run through twice in
sysdeps/mach/hurd/fork.c, but which I can't find start and stop markers
being defined for?


And, might something like that in fact be responsible for the issue I had
already seen months ago, but have just earlier today finally posted as
<http://www.bddebian.com:8888/~hurd-web/open_issues/fork_deadlock/>?


Lastly, _hurd_fork_setup_hook is only defined but never used, it seems.


Grüße,
Thomas

[Svante Signell]

On Thu, 2012-05-10 at 18:09 +0800, Thomas Schwinge wrote:

> And, might something like that in fact be responsible for the issue I had
> already seen months ago, but have just earlier today finally posted as
> <http://www.bddebian.com:8888/~hurd-web/open_issues/fork_deadlock/>?

Thomas, I took the liberty to change "fork will hand" to "fork will
hang" for readability. I hope you don't mind.

[Samuel Thibault]

Thomas Schwinge, le Thu 10 May 2012 15:18:22 +0800, a écrit :
> refs/top-bases/tschwinge/Roger_Whittaker was updated on Savannah, but not
> tschwinge/Roger_Whittaker. This might have happened if the former was
> pushed first and then the connection terminated, for example.

That's possible. tg takes ages and my internet connection went down,
retried, etc. and at some point I must have forgotten about it going on
and not having actually finished it. tg is really hackish :)

Samuel

[Samuel Thibault]

Thomas Schwinge, le Thu 10 May 2012 18:09:03 +0800, a écrit :
> How did this work before, though?

I'd say it didn't.

> Due to __symbol_set_attribute
> specifying weak linkage for the static case, I can see why we didn't get
> undefined symbol errors when linking for the static case, but what about
> the dynamic case? (And that'd mean the whole loop just was a no-op
> before, right?)

Yes.

> Did you find this while working on a specific problem?

I don't remember exactly, it's actually an old commit in my tree.

> Also, what about data_set_element which is only used in hurd/dtable.c for
> _hurd_fork_locks, which is manually run through twice in
> sysdeps/mach/hurd/fork.c, but which I can't find start and stop markers
> being defined for?

data_set_element is something else apparently, but it seems it needs the
same treatment indeed (I had only had a look at the hooks).

> And, might something like that in fact be responsible for the issue I had
> already seen months ago, but have just earlier today finally posted as
> <http://www.bddebian.com:8888/~hurd-web/open_issues/fork_deadlock/>?

Possibly.

> Lastly, _hurd_fork_setup_hook is only defined but never used, it seems.

Possibly a leftover or place for future use, I guess.

Samuel

[Thomas Schwinge]

Hi!

On Fri, 11 May 2012 01:32:52 +0200, Samuel Thibault <samuel....@gnu.org> wrote:
> Thomas Schwinge, le Thu 10 May 2012 18:09:03 +0800, a écrit :
> > Also, what about data_set_element which is only used in hurd/dtable.c for
> > _hurd_fork_locks, which is manually run through twice in
> > sysdeps/mach/hurd/fork.c, but which I can't find start and stop markers
> > being defined for?
>
> data_set_element is something else apparently, but it seems it needs the
> same treatment indeed (I had only had a look at the hooks).
>
> > And, might something like that in fact be responsible for the issue I had
> > already seen months ago, but have just earlier today finally posted as
> > <http://www.bddebian.com:8888/~hurd-web/open_issues/fork_deadlock/>?
>
> Possibly.

Unfortunately that's not it: I've still been seeing that happen with the
hooks patch applied (with _hurd_fork_locks included, too) (Debian eglibc
r5251, but with the libpthread inclusion reverted for the moment). Often
I couldn't get a single GCC build done without having it hang. So, I
wondered why you're not seeing that on the buildds? As I've been
compiling glibc with GCC 4.6, I reverted that to 4.4 which is the Debian
default for glibc. And then I have been doing a lot of successful builds
without any hangs... until this morning, when I again found it hanging at
the very same place. So it's not specifically GCC 4.6's fault either.
Someone will have to look on the critical section locking with a sharp
eye -- especially in conext with Jérémie's signalling patches, as I'm
indicating on the web page.


Grüße,
Thomas

[Thomas Schwinge]

Hi!

On Tue, 24 Sep 2013 23:08:15 +0200, Samuel Thibault <samuel....@ens-lyon.org> wrote:
> 2013-09-24 Samuel Thibault <samuel....@ens-lyon.org>
>
> pthread_atfork needs application callbacks to be called outside any locking.
>
> * sysdeps/mach/hurd/fork.c (_hurd_atfork_prepare_hook, _hurd_atfork_child_hook,
> _hurd_atfork_parent_hook): New hooks.
> (__fork): Call _hurd_atfork_prepare_hook hooks before all locking, call
> _hurd_atfork_parent_hook or _hurd_atfork_child_hook after all unlocking.

This got pushed as commit 7b7bab1391a3b16fff7e325e2c8a36b68eacba90, which
for me causes the glibc build to fail with a linker error, which had also
already been reported twice on IRC (Alicia and Justus CCed), which I in
turn had forgotten about (but had diligently filed in our wiki page
open_issues/pthread_atfork; update is yet to be pushed), until I now hit
it myself:

[...]/libc_pic.os: In function `__fork':
[...]/posix/../sysdeps/mach/hurd/fork.c:70: undefined reference to `__start__hurd_atfork_prepare_hook'
/usr/bin/ld: [...]/libc_pic.os: relocation R_386_GOTOFF against undefined hidden symbol `__start__hurd_atfork_prepare_hook' can not be used when making a shared object
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
make[2]: *** [[...]/libc.so] Error 1

This is the same issue we had before discussed in the thread starting at
<http://news.gmane.org/find-root.php?message_id=%3CE1SMKPZ-0000X2-7c%40vcs.savannah.gnu.org%3E>.

OK to commit the following? We had this locally in t/hooks already, but
it had not been amended for the pthread_atfork symbols.

This Makefile rule is actually only used for the »checking for sufficient
default -shared layout... no« case -- figuring out what that actually is
about is for another day.

2013-12-20 Thomas Schwinge <tho...@codesourcery.com>
Samuel Thibault <samuel....@ens-lyon.org>

* Makerules ($(common-objpfx)shlib.lds): Provide start/stop symbols
for Hurd hooks.

diff --git Makerules Makerules
index 92c2872..35cf552 100644
--- Makerules
+++ Makerules
@@ -482,6 +482,48 @@ $(common-objpfx)shlib.lds: $(common-objpfx)config.make $(..)Makerules
PROVIDE(__start___libc_thread_subfreeres = .);\
__libc_thread_subfreeres : { *(__libc_thread_subfreeres) }\
PROVIDE(__stop___libc_thread_subfreeres = .);\
+ PROVIDE(__start__hurd_fd_subinit = .);\
+ _hurd_fd_subinit : { *(_hurd_fd_subinit) }\
+ PROVIDE(__stop__hurd_fd_subinit = .);\
+ PROVIDE(__start__hurd_subinit = .);\
+ _hurd_subinit : { *(_hurd_subinit) }\
+ PROVIDE(__stop__hurd_subinit = .);\
+ PROVIDE(__start__hurd_proc_subinit = .);\
+ _hurd_proc_subinit : { *(_hurd_proc_subinit) }\
+ PROVIDE(__stop__hurd_proc_subinit = .);\
+ PROVIDE(__start__hurd_pgrp_changed_hook = .);\
+ _hurd_pgrp_changed_hook : { *(_hurd_pgrp_changed_hook) }\
+ PROVIDE(__stop__hurd_pgrp_changed_hook = .);\
+ PROVIDE(__start__hurd_reauth_hook = .);\
+ _hurd_reauth_hook : { *(_hurd_reauth_hook) }\
+ PROVIDE(__stop__hurd_reauth_hook = .);\
+ PROVIDE(__start__hurd_fork_locks = .);\
+ _hurd_fork_locks : { *(_hurd_fork_locks) }\
+ PROVIDE(__stop__hurd_fork_locks = .);\
+ PROVIDE(__start__hurd_atfork_prepare_hook = .);\
+ _hurd_atfork_prepare_hook : { *(_hurd_atfork_prepare_hook) }\
+ PROVIDE(__stop__hurd_atfork_prepare_hook = .);\
+ PROVIDE(__start__hurd_atfork_child_hook = .);\
+ _hurd_atfork_child_hook : { *(_hurd_atfork_child_hook) }\
+ PROVIDE(__stop__hurd_atfork_child_hook = .);\
+ PROVIDE(__start__hurd_atfork_parent_hook = .);\
+ _hurd_atfork_parent_hook : { *(_hurd_atfork_parent_hook) }\
+ PROVIDE(__stop__hurd_atfork_parent_hook = .);\
+ PROVIDE(__start__hurd_fork_prepare_hook = .);\
+ _hurd_fork_prepare_hook : { *(_hurd_fork_prepare_hook) }\
+ PROVIDE(__stop__hurd_fork_prepare_hook = .);\
+ PROVIDE(__start__hurd_fork_setup_hook = .);\
+ _hurd_fork_setup_hook : { *(_hurd_fork_setup_hook) }\
+ PROVIDE(__stop__hurd_fork_setup_hook = .);\
+ PROVIDE(__start__hurd_fork_child_hook = .);\
+ _hurd_fork_child_hook : { *(_hurd_fork_child_hook) }\
+ PROVIDE(__stop__hurd_fork_child_hook = .);\
+ PROVIDE(__start__hurd_fork_parent_hook = .);\
+ _hurd_fork_parent_hook : { *(_hurd_fork_parent_hook) }\
+ PROVIDE(__stop__hurd_fork_parent_hook = .);\
+ PROVIDE(__start__hurd_preinit_hook = .);\
+ _hurd_preinit_hook : { *(_hurd_preinit_hook) }\
+ PROVIDE(__stop__hurd_preinit_hook = .);\
/DISCARD/ : { *(.gnu.glibc-stub.*) }@'
test -s $@T
mv -f $@T $@


Grüße,
Thomas

[Roland McGrath]

The linker is supposed to define those symbols automagically.
Did it break?

[Samuel Thibault]

Hello,

Roland McGrath, le Fri 20 Dec 2013 13:05:42 -0800, a écrit :
> The linker is supposed to define those symbols automagically.
> Did it break?

We have been having

“

checking for sufficient default -shared layout... no

”

at least since 2.17, yes. (I don't have logs between 2.13 and 2.17)

Note that this is a Debian build, and also happens on Linux ports.

Here is the config.log output from 2.17:

configure:6644: checking for sufficient default -shared layout
configure:6663: i486-gnu-gcc-4.7 -shared -o conftest.so conftest.s 1>&5
configure:6666: $? = 0
nono from:

NOTE .note.gnu.build-i
GNU_HASH .gnu.hash
DYNSYM .dynsym
STRTAB .dynstr
VERSYM .gnu.version
VERNEED .gnu.version_r
REL .rel.dyn
REL .rel.plt
PROGBITS .init
PROGBITS .plt
PROGBITS .text
PROGBITS .fini
NOTE .note.a
NOTE .note.b
PROGBITS .eh_frame
INIT_ARRAY .init_array
FINI_ARRAY .fini_array
PROGBITS .jcr
DYNAMIC .dynamic
PROGBITS .got
PROGBITS .got.plt
PROGBITS .data
NOBITS .bss
PROGBITS .comment
STRTAB .shstrtab
SYMTAB .symtab
STRTAB .strtab
configure:6698: result: no

So the NOTEs are indeed not put at the beginning by the debian linker.

Samuel

[Roland McGrath]

> Roland McGrath, le Fri 20 Dec 2013 13:05:42 -0800, a �,Aicrit :

> > The linker is supposed to define those symbols automagically.
> > Did it break?
>
> We have been having
>

> �1r|

> checking for sufficient default -shared layout... no

What does this have to do with anything?

Are you saying that the linker does not define __start_* and __stop_*
symbols automagically when it's given a custom linker script?

[Samuel Thibault]

Roland McGrath, le Mon 03 Mar 2014 15:36:52 -0800, a écrit :
> > Roland McGrath, le Fri 20 Dec 2013 13:05:42 -0800, a ,Aicrit :
> > > The linker is supposed to define those symbols automagically.
> > > Did it break?
> >
> > We have been having
> >

> > $,1r|

> > checking for sufficient default -shared layout... no
>
> What does this have to do with anything?

It is what makes the Makefile choose between using the empty
shlib-lds/shlib-lds-flags or using the explicit shlib.lds file.

> Are you saying that the linker does not define __start_* and __stop_*
> symbols automagically when it's given a custom linker script?

It seems libc's __start_* and __stop_* symbols get defined automagically
indeed. This however doesn't happen for libpthread apparently:

/usr/src/eglibc-2.18/build-tree/hurd-i386-libc/libpthread/libpthread_pic.a(pt-initialize.os): In function `__pthread_init':
/usr/src/eglibc-2.18/libpthread/./pthread/pt-initialize.c:77: undefined reference to `__stop___pthread_init'
/usr/bin/ld.bfd.real: /usr/src/eglibc-2.18/build-tree/hurd-i386-libc/libpthread/libpthread_pic.a(pt-initialize.os): relocation R_386_GOTOFF against undefined hidden symbol `__stop___pthread_init' can not be used when making a shared object

Now I'm wondering why for instance __start/stop___libc_atexit are
specified. This comes from Ulrich's “Place __libc_subfreeres,
__libc_atexit, and __libc_thread_subfreeres sections after .jcr
section.”. I don't know what this is really useful for.

Samuel