shared libraries

[Ben Gras]

Hey guys,

People are invited to test shared libraries. Some usage and implementation details are here:

http://wiki.minix3.org/en/UsersGuide/UsingSharedLibraries

And also invited to help fix package building with shared libraries (minix-master-dynamic branch in pkgsrc).

[Evgeniy Ivanov]

Wow! Nice and hard work!

> --
> You received this message because you are subscribed to the Google Groups
> "minix3" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/minix3/-/UY8gsQYz1VMJ.
> To post to this group, send email to min...@googlegroups.com.
> To unsubscribe from this group, send email to
> minix3+un...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/minix3?hl=en.



--
Evgeniy

[Piotr Tworek]

Hi,

I've been using this code from the day it was committed and I must say
that so far it works fine. Thank you for your work on this.

As for the shared library support in pkgsrc most of the work needed seem
to be related to libtool. The libtool.m4 file needd to be modified so
that configure scripts generated from it will detect minix shared
library support. Ideally the patches should be sent upstream so at some
point new releases of projects using autotools will support minix out of
the box. I'm not much of a libtool expert but I've tried to prepare such
patches and result can be found here [1]
(devel/libtool/patches/patch-minix-{libtool|ltdl}.m4). I've also started
patching some other packages to detect that minix supports shared
libraries. Please feel free to merge them info the minix-master-dynamic
branch if you feel they are useful. The resulting shared libraries work.
Currently I'm using gcc binary that is dynamically linked to both gmp
and mpfr and git binary that is dynamically linked with openssl. No
problems so far.

Besides that would it be a problem to also build libarchive from
minix.git repo as shared library? This [2] patch can be used for this.

/ptw

[1] https://github.com/tworaz/minix-pkgsrc/commits/minix-shlib-work
[2]
https://github.com/tworaz/minix/commit/c4ac3e8c115df7389ab4cdc7253f3b9401b933ca

[Antoine LECA]

Ben Gras wrote:
> People are invited to test shared libraries. Some usage and implementation
> details are here:
>
> http://wiki.minix3.org/en/UsersGuide/UsingSharedLibraries

GREAT! Congratulations, I know it was not easy ;-)

I only tested it lightly, but so far it work smoothly... even while
trying strange things like using a text symbol as data :-)

There is something which is surprising me however, it is the use of the
shlib number of NetBSD (i.e. /lib/libc.so.12.175).
We are incompatible with NetBSD on two fundamental issues:
- some constants are different (SIGVTALRM, ELOOP)
- some typedefs are different (off_t, time_t)
I see those issues as fundamental because it impedes the exchange of
binaries between the two operating systems; OTOH, the lack of some
system calls in MINIX is not no serious: a NetBSD binary which is
calling setpgid() won't be loaded on MINIX since there is no shared
objects which provides this symbol: this clearly prevents bad things to
occur; but if lseek(2) is called with the wrong size, chaos will result.

Of course, this was not an issue at all with static linking, but shared
libraries also bring in their own problems :-) (this is known, in a
slightly different form, in the Windows world as "DLL hell".)

Now, such issues are dealt with upgrades of the standard C library;
NetBSD did it in the past, particularly for typedefs: this is why there
are all those _time50 "renamed" symbols (which is the 32-bit time_t
variant, present up to NetBSD 5.)
The easiest counterpart of such changes, as I understand it, is that the
major number (.12) is bumped, to avoid possible mismatches: the ELF
standard prevents the dynamic linker to successfully bind to a shared
object is the asked-for major number differ. Obviously, this easy scheme
will not work with the shared libraries, because of the use of NetBSD
numbers... unless we change it to some lesser number NOW, before any
package are distributed in the wild!

Then, assuming we want to be able to use directly NetBSD binaries (which
seem to me a welcome idea, why would you recompile Gcc or Firefox if you
can use the i386 NetBSD binary directly?), we should amend all the
incompatible 'fundamental' constants and typedefs to match those of
NetBSD. Of course each of these changes will mean a difficult upgrade to
do: which is why I advocate to start sooner! Perhaps it will resolve as
a "big bang switch" for the vast majority of people, such as what was
the switch to the new libc, where the compatibility period (when the new
libc was compilable with the non-ELF gcc), or the new boot loader, came
to an end very quickly because it was not sustainable to maintain both
lines.


I do not want here to paint in black the work done, while I am really
impressed! But I thought the problem I forecast is real, and will only
cause griefs in the future if not handled now. Please consider it.


Antoine

[Ben Gras]

all,

I had replied already by email to /ptw, but just so that everyone knows how great this work is: it is very great.

thomas cort took a look and merged it with the minix-dynamic branch.

thanks a lot for putting all that work in, and to TC for merging it. it makes the goal of having all-dynamic packages a whole lot closer!

[Ben Gras]

Hi antoine,

thanks for your enthusiasm :).

personally i have my doubts about wanting to run netbsd binaries. it seems to me effort put towards running netbsd binaries (being abi-compatible) is better put towards being source-compatible.

i admit it's not such a crazy idea to think about future compatability if it's little effort now. but as long as the libraries are abi-incompatible with applications (errno's, signal numbers, types, etc.), they won't work anyway, right version or not.

it seems to me it's so much work to make the two systems abi-compatible (abi incompatabilities already mentioned; plus adding the netbsd system call interface to minix) that they probably will bump their major version by then and we can just target that.

btw it is a goal to harmonize the types (64-bit off_t and such).

[Antoine LECA]

Ben Gras wrote:
> i admit it's not such a crazy idea to think about future compatability if
> it's little effort now. but as long as the libraries are abi-incompatible
> with applications (errno's, signal numbers, types, etc.), they won't work
> anyway, right version or not.

My point was that I fail to see anything in the "etc." other than IOCTL
constants and perhaps a few more constants; and the very fact we are
using a different set of constants for semi-hidden stuff is IMHO a
maintenance problem expecting to bite us in the future.

But I can easily have missed something here: please advise.

> it seems to me it's so much work to make the two systems abi-compatible
> (abi incompatabilities already mentioned; plus adding the netbsd system
> call interface to minix)

Huh? The beauty of dynamic libc.so is precisely that the details of the
implementation of the system call interface is hidden within the libc.so
This is the very point of OS/2|Windows DLLs, and the very point of libc6
a.k.a GNU libc in Linux (combined with the former switch from jumptables
to ELF to enable dynamic linking): they are creating a standard ABI
which only consists of ordinary-looking function calls, bound by name at
execution time to the actual implementations.

For example, the fact a system call is implemented as some i386 software
INT, or as a call gate, or using the fast switching mechanism, using one
or other system call numbers system, is completely transparent to the
application binaries which link to libc.so; as is any evolution from one
mechanism to another.

> that they probably will bump their major version
> by then and we can just target that.

For the record: I doubt NetBSD will change their libc.so major version
up: it is such since 1997 (1.3). See for example the CVS comment
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/shlib_version#rev1.44


Antoine

[Ben Gras]

All,

On Wednesday, May 2, 2012 1:37:23 PM UTC+1, AntoineLeca wrote:

Ben Gras wrote:
> i admit it's not such a crazy idea to think about future compatability if
> it's little effort now. but as long as the libraries are abi-incompatible
> with applications (errno's, signal numbers, types, etc.), they won't work
> anyway, right version or not.

My point was that I fail to see anything in the "etc." other than IOCTL
constants and perhaps a few more constants; and the very fact we are
using a different set of constants for semi-hidden stuff is IMHO a
maintenance problem expecting to bite us in the future.

But I can easily have missed something here: please advise.

I'm not sure. I'm sure lots of nasty details will keep coming like from a clown car. Because that's been my experience with all other such 'square peg round hole' situations. Never insurmountable but always underestimated and unforeseen, and each one can be painful to debug.
 

Huh? The beauty of dynamic libc.so is precisely that the details of the
implementation of the system call interface is hidden within the libc.so
This is the very point of OS/2|Windows DLLs, and the very point of libc6
a.k.a GNU libc in Linux (combined with the former switch from jumptables
to ELF to enable dynamic linking): they are creating a standard ABI
which only consists of ordinary-looking function calls, bound by name at
execution time to the actual implementations.

Good point! I hadn't thought of it this way.
That does make the executables neatly much-more-OS-independent.

Honestly, the problem is: thinking about such binary compatability now, to save trouble in the future *useful only after an unpredictable amount of extra work (see above)*, for uncertain reward (to my mind the reward is more esthetic than practical - we're just saving recompiling) is extremely touchy.

If it's a matter of decrementing the major version(s) now, then I think that could be a good idea (good time/future disaster tradeoff). As the base system isn't built dynamically by default yet, and there are no dynamic packages out there, we can do this without fallout (things breaking). And we can increase the major version once the calls are abi-compatible. Perhaps you'd be motivated to figure out which libraries need such treatment?

[Antoine LECA]

Ben Gras wrote:
> If it's a matter of decrementing the major version(s) now, then I think
> that could be a good idea (good time/future disaster tradeoff). As the base
> system isn't built dynamically by default yet, and there are no dynamic
> packages out there, we can do this without fallout (things breaking).

That was entirely my point: I do not underestimate the likelihood to
never successfully mix MINIX and NetBSD binaries on the same basic
system (without emulation layer, that is), I was just pointing out that
we should not slap that door; and unlinking the libc.so numbers at the
moment is a right step at the moment. IMHO, of course. ;-)

> And we can increase the major version once the calls are abi-compatible.

Yes, my idea exactly.

> Perhaps you'd be motivated to figure out which libraries need such
> treatment?

Very good point, and sometimes I have not decided. I guess there are
several subcases, with some which are unaffected at all (things like
libarchive perhaps); things that are basically affected only because of
some typedef (like are off_t or time_t); and some which are at the same
low-level as libc is (like libcompat perhaps?)
Just guesses; there is a definitive need to more investigation here.


BTW this is something which many people could opine about; I would
particularly like to hear people with a long background on version
handling in other operating systems.


Antoine

[pikpik]

Hi,

On Apr 25, 10:12 am, Ben Gras wrote:

> Hey guys,
>
> People are invited to test shared libraries. Some usage and implementation
> details are here:

This is awesome! Thank you very much for doing this!

pikpik

[Antoine LECA]

Ben Gras wrote:
> People are invited to test shared libraries.

I do not know if this is a bug or a feature; but I observe from
preliminary experience that the old a.out binaries (both the A_SEP
created with ACK, and the "com" created with gcc up to 4.1) are not
working any more with the new changes introduced with shared libraries.
The error returned is "not found" (ENOENT?) No message at console.


In case it is not already obvious: this seems to me a major break in the
backward compatibility: basically, anything compiled for MINIX up to
3.1.9 will not work any more on future systems.
This also means that while upgrading, 3.2.0 in a mandatory stop.


Antoine

[Antoine LECA]

I (Antoine LECA) wrote:
> I do not know if this is a bug or a feature; but I observe from
> preliminary experience that the old a.out binaries (both the A_SEP
> created with ACK, and the "com" created with gcc up to 4.1) are not
> working any more with the new changes introduced with shared libraries.
> The error returned is "not found" (ENOENT?) No message at console.

It really looks like a bug: the newly introduced elf_has_interpreter()
function, which returns 0 if NO, 1 if YES and negative values on error,
is used in a raw if() on line 272 of vfs/exec.c; the attached patch
corrects this (and the similar case in RS, although I did not test it).

You can also cherry-pick 48fd0fe on my github repository.


Antoine