Fwd: Warnings about dlclose before thread exit. __cxa_thread_call_dtors

[Willem Jan Withagen]

Hi,

I've sent the question below to the Ceph-devel list, asking if any
recent changes would be able to cause this.

But then of course this could stem from FreeBSD libs, and of ports....
So the question here is if anybody has gotten these "warnings" in other
tools.

--WjW


-------- Forwarded Message --------
Subject: Warnings about dlclose before thread exit
Date: Sat, 18 Aug 2018 14:46:35 +0200
From: Willem Jan Withagen <w...@digiware.nl>
To: Ceph Development <ceph-...@vger.kernel.org>

Hi,

I've have upgraded to FreeBSD ALPHA 12.0, but I don't think the errors
them from there. Although they could be in one of the libs that came
along with the upgrade.

I'm getting these warnings during rbd and ceph (maybe even more)
invocations that indicate that indicate a possible problem because:
===
It could be possible that a dynamically loaded library, use
thread_local variable but is dlclose()'d before thread exit. The
destructor of this variable will then try to access the address,
for calling it but it's unloaded, so it'll crash. We're using
__elf_phdr_match_addr() to detect and prevent such cases and so
prevent the crash.
===
this is from :
https://github.com/freebsd/freebsd/blob/master/lib/libc/stdlib/cxa_thread_atexit_impl.c

Now it could be that dlclose() and thread exit are just closed to one
another. But still this is hard core embedded in libc already since
2017, so I'm sort of expecting that a recent change has caused this.

And as indicated it is a possible cause for crashed, because thread_exit
is going to clean up things that are no longer there.

Now the 20 dollar question is:
Where was this introduced??

Otherwise I'll have to try and throw my best gdb capabilities at it, and
try to invoke an rbd call and see where it activates this warning.

--WjW
_______________________________________________
freebsd...@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-curre...@freebsd.org"

[David Chisnall]

The FreeBSD implementation here looks racy. If one thread dlcloses an object while another thread is exiting, we can end up calling a function at an invalid memory address. It also looks as if it may be possible to unload one library, load another at the same address, and end up executing entirely the wrong code, which would have some serious security implications.

The GNU/Linux equivalent of this function locks the DSO in memory until all references to it have gone away. A call to dlclose() on GNU/Linux will not actually unload the library until all threads with destructors in that library have been unloaded. I believe that this reuses the same reference counting mechanism that allows the same library to be dlopened and dlclosed multiple times.

It would be nice if the FreeBSD version had the same behaviour, because this is almost certainly expected in code written on other platforms.

David

[Willem Jan Withagen]

On 26/08/2018 12:19, David Chisnall wrote:
> The FreeBSD implementation here looks racy. If one thread dlcloses an object while another thread is exiting, we can end up calling a function at an invalid memory address. It also looks as if it may be possible to unload one library, load another at the same address, and end up executing entirely the wrong code, which would have some serious security implications.
>

The error report occurs when the thread terminates. And gdb really
points me to the destructor of a class. (Random in this case)

> The GNU/Linux equivalent of this function locks the DSO in memory until all references to it have gone away. A call to dlclose() on GNU/Linux will not actually unload the library until all threads with destructors in that library have been unloaded. I believe that this reuses the same reference counting mechanism that allows the same library to be dlopened and dlclosed multiple times.
>
> It would be nice if the FreeBSD version had the same behaviour, because this is almost certainly expected in code written on other platforms.

ehh, yes, ...

For the moment I disabled the notice.. It drives me crazy.
--WjW