Static System Libraries in Solaris 10
John Barton
This announcement applies only to 32-bit static system libraries and
statically linked utilites. 64-bit static system libraries and
utilities have never been provided.
The Solaris Application Binary Interface (ABI) is provided through
dynamic system libraries. All Solaris applications should be linked
with the Solaris dynamic system libraries. 32-bit applications linked
with Solaris static system libraries do not utilize the Solaris ABI.
Support for 32-bit Solaris static system libraries and statically
linked utilities will not be provided in a future release. Of
particular note, support for the static C library (/usr/lib/libc.a)
will not be provided in a future release.
Please send feedback/comments to: static-lib...@sun.com
Drazen Kacar
> Support for 32-bit Solaris static system libraries and statically
> linked utilities will not be provided in a future release. Of
Hm. The message body says "future release", but the subject says "Solaris
10". Can I take the subject as a normative reference? :-)
--
.-. .-. I don't work here. I'm a consultant.
(_ \ / _)
| da...@willfork.com
|
Rich Teer
> Support for 32-bit Solaris static system libraries and statically
> linked utilities will not be provided in a future release. Of
> particular note, support for the static C library (/usr/lib/libc.a)
> will not be provided in a future release.
I'm all for losing the static libraries, but I think that
the statically linked binaries (currently in /usr/sbin/static)
should be kept - you never know when they will be handy.
In fact, they've saved my skin at least once.
Actually, if anything, those binaries should be moved to
/sbin/static, to be of use to people who insist on using a
separate /usr partition.
In summary:
1. Lose the static libraries as planned
2. Move the static binaries in /usr/sbin/static to /sbin/static
That's my CDN $0.02...
--
Rich Teer
President,
Rite Online Inc.
Voice: +1 (250) 979-1638
URL: http://www.rite-online.net
Akop Pogosian
> On 31 Oct 2002, John Barton wrote:
>> Support for 32-bit Solaris static system libraries and statically
>> linked utilities will not be provided in a future release. Of
>> particular note, support for the static C library (/usr/lib/libc.a)
>> will not be provided in a future release.
> I'm all for losing the static libraries, but I think that
> the statically linked binaries (currently in /usr/sbin/static)
> should be kept - you never know when they will be handy.
> In fact, they've saved my skin at least once.
> Actually, if anything, those binaries should be moved to
> /sbin/static, to be of use to people who insist on using a
> separate /usr partition.
> In summary:
> 1. Lose the static libraries as planned
> 2. Move the static binaries in /usr/sbin/static to /sbin/static
But.. how can Sun continue supporting the static binaries if they will
discontinue support for static libraries?
--
Akop Pogosian
This space has been accidentally left blank.
Dave Uhring
> In comp.unix.solaris Rich Teer <ri...@rite-group.com> wrote:
>> On 31 Oct 2002, John Barton wrote:
>
>>> Support for 32-bit Solaris static system libraries and statically
^^^^^^^^^^^^^^
>>> linked utilities will not be provided in a future release. Of
^^^^^^^^^^^^^^^^
>>> particular note, support for the static C library (/usr/lib/libc.a)
>>> will not be provided in a future release.
>
>> I'm all for losing the static libraries, but I think that
>> the statically linked binaries (currently in /usr/sbin/static)
>> should be kept - you never know when they will be handy.
>> In fact, they've saved my skin at least once.
>
>> Actually, if anything, those binaries should be moved to
>> /sbin/static, to be of use to people who insist on using a
>> separate /usr partition.
>
>> In summary:
>
>> 1. Lose the static libraries as planned
>
>> 2. Move the static binaries in /usr/sbin/static to /sbin/static
>
> But.. how can Sun continue supporting the static binaries if they will
> discontinue support for static libraries?
>
Heh! Neither one of you is reading the article ;-)
It looks like /usr better be on the / partition from that point on.
Rich Teer
> But.. how can Sun continue supporting the static binaries if they will
> discontinue support for static libraries?
Sun has the full source code. What they use internally,
and what they make available externally don't necessarily
have to be the same thing...
Tim Bradshaw
> Sun has the full source code. What they use internally,
> and what they make available externally don't necessarily
> have to be the same thing...
However, it would be nice if a user of the system can *also* construct
static binaries, if needed.
--tim
Barry Margolin
Would you be willing to live with the caveat that they would have to be
relinked any time a library patch is installed or the OS is upgraded?
Maintaining upward compatibility of the library API is relatively easy;
maintaining binary compatibility is much harder, and may limit what the
vendor can do to fix bugs.
--
Barry Margolin, bar...@genuity.net
Genuity, Woburn, MA
*** DON'T SEND TECHNICAL QUESTIONS DIRECTLY TO ME, post them to newsgroups.
Please DON'T copy followups to me -- I'll assume it wasn't posted to the group.
Tim Bradshaw
> Would you be willing to live with the caveat that they would have to be
> relinked any time a library patch is installed or the OS is upgraded?
> Maintaining upward compatibility of the library API is relatively easy;
> maintaining binary compatibility is much harder, and may limit what the
> vendor can do to fix bugs.
I don't really understand this. If a binary is statically linked,
then surely the only calls it makes are system calls? I don't think
they get changed all the time. Sure, if it uses a lot of libraries
that do stuff which exchange data whose format might change (so IPC of
whatever kind) hidden behind a library, then it might lose, but how
common is that?
In fact, in the early days of what is now xemacs (then lemacs) I
remember that a lot of the binary distributions that got made were
statically linked, because that way you only depended on low-level
stuff like the X protocol and system calls, not which of the 983
versions of some random library happened to be installed on the target
machine. This was done because there was *so* much trouble from
dynamic binaries that just wouldn't run.
*However* it looks (from reading the original article) as if Sun are
intending to not provide any static binaries at all, not even /sbin/sh
(say). My real issue is that I don't want there to be (too many)
magic things that Sun can produce (static binaries) which I can't.
--tim
Casper H.S. Dik
>I don't really understand this. If a binary is statically linked,
>then surely the only calls it makes are system calls? I don't think
>they get changed all the time. Sure, if it uses a lot of libraries
>that do stuff which exchange data whose format might change (so IPC of
>whatever kind) hidden behind a library, then it might lose, but how
>common is that?
They do get changed some of the time; some system calls are added;
others have been removed or changed, sometimes in incompatible ways.
>*However* it looks (from reading the original article) as if Sun are
>intending to not provide any static binaries at all, not even /sbin/sh
>(say). My real issue is that I don't want there to be (too many)
>magic things that Sun can produce (static binaries) which I can't.
No static binaries, correct.
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
Barry Margolin
Casper H.S. Dik <Caspe...@Sun.COM> wrote:
>Tim Bradshaw <t...@cley.com> writes:
>
>>I don't really understand this. If a binary is statically linked,
>>then surely the only calls it makes are system calls? I don't think
>>they get changed all the time. Sure, if it uses a lot of libraries
>>that do stuff which exchange data whose format might change (so IPC of
>>whatever kind) hidden behind a library, then it might lose, but how
>>common is that?
>
>They do get changed some of the time; some system calls are added;
>others have been removed or changed, sometimes in incompatible ways.
Like when the socket API was changed from system calls to Streams-based
library functions.
These types of incompatible changes at the binary level are probably pretty
infrequent in patches, but I'll bet there are a few of them during each
major release of the OS.
Darren Dunham
news:3dc2eafa$0$46604$e4fe...@news.xs4all.nl:
> Tim Bradshaw <t...@cley.com> writes:
>>*However* it looks (from reading the original article) as if Sun are
>>intending to not provide any static binaries at all, not even /sbin/sh
>>(say). My real issue is that I don't want there to be (too many)
>>magic things that Sun can produce (static binaries) which I can't.
>
> No static binaries, correct.
So does that include /sbin/mount? Does that mean that /usr cannot be on a
separate (and possibly read-only) partition?
--
Darren Dunham
ddu...@taos.com
grog
john....@sun.com says...
Why? What, exactly, is the problem with the static libs? So the static
32-bit apps don't use the Solaris ABI, who cares. Get too restrictive
and try to force everybody into the "Sun way" and you'll start to drop
to the dark depths of Microsoft.
Casper H.S. Dik
>Why? What, exactly, is the problem with the static libs? So the static
>32-bit apps don't use the Solaris ABI, who cares. Get too restrictive
>and try to force everybody into the "Sun way" and you'll start to drop
>to the dark depths of Microsoft.
Even in the current situtation, there have been many problems with
static libraries:
- linking statically against libsocket/libnsl
- broken when upgrading to 2.6
- partially static linking (libc statically, lib thread not)
- applications broke when the libthread/libc interfaces
got upgraded
It's especially the ability to pick and chose libraries that gives no
end to trouble.
And Solaris 10 adds another very good reason; it pushed the static
libraries off the edge.
Drazen Kacar
> - partially static linking (libc statically, lib thread not)
> - applications broke when the libthread/libc interfaces
> got upgraded
It's been like that for years, so why did it become a problem worth solving
now? Hmmm... TLS alone can't be the reason... But perhaps you're going to
make dynamic loading of libthread (and thus making program "threaded" on
the fly) legal and supported?
rachel polanskis
<Pine.GSO.4.44.021031...@grover.rite-group.com>,
Rich Teer <ri...@rite-group.com> wrote:
> On 31 Oct 2002, John Barton wrote:
>
> > Support for 32-bit Solaris static system libraries and statically
> > linked utilities will not be provided in a future release. Of
> > particular note, support for the static C library (/usr/lib/libc.a)
> > will not be provided in a future release.
>
> I'm all for losing the static libraries, but I think that
> the statically linked binaries (currently in /usr/sbin/static)
> should be kept - you never know when they will be handy.
> In fact, they've saved my skin at least once.
>
> Actually, if anything, those binaries should be moved to
> /sbin/static, to be of use to people who insist on using a
> separate /usr partition.
>
> In summary:
>
> 1. Lose the static libraries as planned
>
> 2. Move the static binaries in /usr/sbin/static to /sbin/static
I agree - the static binaries have saved me more than once too, as I
have root mounted as its own filesystem, with /usr on another.
I do not like the idea of having to mount /root and /usr on the same
filesystem. I would concede to having shared objects rather than static
if there was a root mounted /lib with a subset of the necessary shared
objects required to boot a system in distress and repair it.
Currently, /lib is a symlink to /usr/lib which is useless if you want to
be able to boot the system and /usr is unmountable. I am, obviously of
the school that doesn't like the critical filesystems all mounted under
/root.
rachel
Richard L. Hamilton
Seems a copy of libc.so.1 (albeit linked a bit differently?) in /etc/lib
would take care of it. That means root needs to be about 1 MB larger.
If a few more libraries were needed, that might grow to 2 MB larger. OTOH,
you'd get some of that back, inasmuch as dynamically linked binaries are
smaller than statically linked ones.
Under that scenario, the only greater risk I see is that a few essential
statically linked binaries might be viewed as a "smaller target" to
filesystem damage than similar dynamically linked binaries plus the
libraries they were linked to.
--
mailto:rlh...@mindwarp.smart.net http://www.smart.net/~rlhamil
Joerg Schilling
Richard L. Hamilton <rlh...@smart.net> wrote:
>Seems a copy of libc.so.1 (albeit linked a bit differently?) in /etc/lib
>would take care of it. That means root needs to be about 1 MB larger.
>If a few more libraries were needed, that might grow to 2 MB larger. OTOH,
>you'd get some of that back, inasmuch as dynamically linked binaries are
>smaller than statically linked ones.
The only problem may be that you will not be able to replace /usr/lib/libc.so*
if you neither have a static "cp", "mv" nor "tar".
--
EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
j...@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schi...@fokus.gmd.de (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Alan Stange
OK. Looks like some folks have their undies in a bundle because of the
"no static binaries" feature of Solaris ++9.
As Casper mentioned, this doesn't mean that you can't have a separate
/usr partition. In the simplest case, the / partition has a /usr/lib
directory in it that contains a libc.so.
Do what you want at boot time, etc. Now, mount /usr and the underlying
/usr/lib is now buried below the UFS mount. What am I missing?
The question I have is what features are gained by this?
-- Alan
Chris Thompson
Alan Stange <sta...@rentec.com> wrote:
>
>OK. Looks like some folks have their undies in a bundle because of the
>"no static binaries" feature of Solaris ++9.
>
>As Casper mentioned, this doesn't mean that you can't have a separate
>/usr partition. In the simplest case, the / partition has a /usr/lib
>directory in it that contains a libc.so.
>
>Do what you want at boot time, etc. Now, mount /usr and the underlying
>/usr/lib is now buried below the UFS mount. What am I missing?
The ability to patch the root filing system versions of /usr/lib while
the system is running!
There are dynamically linked binaries in /sbin that are executable before
/usr is mounted in existing releases of Solaris. They use the shared
libraries in /etc/lib. I expect beefing up the latter is what Sun have
in mind.
Chris Thompson
Email: cet1 [at] cam.ac.uk
Richard L. Hamilton
For the end user, not much, aside from avoiding version skew given that
patches to shared libc and to static libc and static binaries are rarely
simultaneous. For Sun, not needing to maintain static libc and statically
linked binaries, and a bit more freedom to change internals without
worrying about what will break (as long as APIs/ABIs are kept consistent),
which in turn reduces the cost (and overhead in obsolete code) of keeping
binary compatibility.
Richard L. Hamilton
ce...@cus.cam.ac.uk (Chris Thompson) writes:
> In article <3dc6ba4f$0$26649$724e...@reader2.ash.ops.us.uu.net>,
> Alan Stange <sta...@rentec.com> wrote:
>>Do what you want at boot time, etc. Now, mount /usr and the underlying
>>/usr/lib is now buried below the UFS mount. What am I missing?
>
> The ability to patch the root filing system versions of /usr/lib while
> the system is running!
NFS can get at a directory hidden under a mount point. But that's
something to keep in mind if one suspects a file is there that shouldn't
be, not a way around your point, since it's quite impractical (and
insecure, unless restricted to themselves only) to require everyone to
share /.
> There are dynamically linked binaries in /sbin that are executable before
> /usr is mounted in existing releases of Solaris. They use the shared
> libraries in /etc/lib. I expect beefing up the latter is what Sun have
> in mind.
That's what I'm guessing too.
Casper H.S. Dik
>In article <usap86h...@corp.supernews.com>,
>Richard L. Hamilton <rlh...@smart.net> wrote:
>>Seems a copy of libc.so.1 (albeit linked a bit differently?) in /etc/lib
>>would take care of it. That means root needs to be about 1 MB larger.
>>If a few more libraries were needed, that might grow to 2 MB larger. OTOH,
>>you'd get some of that back, inasmuch as dynamically linked binaries are
>>smaller than statically linked ones.
>The only problem may be that you will not be able to replace /usr/lib/libc.so*
>if you neither have a static "cp", "mv" nor "tar".
Sure you can.
"ln" (ln libc.so.1 libc.so.old; ln libc.so.new libc.so.1)
"cpio" (will do the rigth thing when overwriting a file;
create /usr/lib/XXXXX and rename)
and then there's:
LD_LIBRARY_PATH=/emergency/lib /emergency/lib/ld.so.1 <cmd> ...
Joerg Schilling
Casper H.S. Dik <Caspe...@Sun.COM> wrote:
>>The only problem may be that you will not be able to replace /usr/lib/libc.so*
>>if you neither have a static "cp", "mv" nor "tar".
>
>Sure you can.
>
>"ln" (ln libc.so.1 libc.so.old; ln libc.so.new libc.so.1)
Correct, I forget about this because it is new to auto remove the ln
target IIRC
>"cpio" (will do the rigth thing when overwriting a file;
> create /usr/lib/XXXXX and rename)
Mmmm why put effort into a command that has been removed from the POSIX
standard?
>and then there's:
>
>
> LD_LIBRARY_PATH=/emergency/lib /emergency/lib/ld.so.1 <cmd> ...
If Sun supplies this emergency lib it looks like a nice idea.
Roger Marquis
>For Sun, not needing to maintain static libc and statically
>linked binaries, and a bit more freedom to change internals without
>worrying about what will break (as long as APIs/ABIs are kept consistent),
>which in turn reduces the cost (and overhead in obsolete code) of keeping
>binary compatibility.
The cost of maintaining static binaries, both in developer and user
hours, has been well out of proportion to their usefulness for over
a decade now. Sun is just following the lead of other Unix
distributions which work well without static binaries.
Good to see Solaris engineering looking ahead and eliminating more
legacy-only code (though I sure miss the easy to configure Openwindows
menu).
Hope the trend continues with the elimination of another needless
complexity that also causes more problems than it solves: /usr
partitions.
--
Roger Marquis
Roble Systems Consulting
http://www.roble.com/
Casper H.S. Dik
>> LD_LIBRARY_PATH=/emergency/lib /emergency/lib/ld.so.1 <cmd> ...
>If Sun supplies this emergency lib it looks like a nice idea.
Well, it means that you can recover from
mv /usr/lib /usr/lib.old
without static utilities, even now.
Rich Teer
> Hope the trend continues with the elimination of another needless
> complexity that also causes more problems than it solves: /usr
> partitions.
Partitioning /usr separately from / has been optional for
many years. I tried it once, and concurr withyour opinion!
Rich Teer
> However, if you want to use ufs snapshots on /usr you need it seperate
> from /
That's not been a problem for me (after stopping xntpd).
Chuck Swiger
[ ... ]
I don't have any problem with Sun removing static libraries from Solaris so
long as they keep up the ABI compatibility-- as they have done, and done well,
in the past.
> The cost of maintaining static binaries, both in developer and user
> hours, has been well out of proportion to their usefulness for over
> a decade now. Sun is just following the lead of other Unix
> distributions which work well without static binaries.
For a counterexample, look up John Dyson's remarks on the compilation time of
"make world" under FreeBSD using a staticly linked /bin/sh, make, and compiler
subsystem, versus that of dynamicly linked system tools.
Dynamic linking adds a very significant amount of overhead to process startup
times and in terms of memory bloat for dynamic linking (per process link
tables, memory objects probably requiring page alignment, etc). For that
matter, are ELF binaries page aligned on disk, taking up more space?
> Hope the trend continues with the elimination of another needless
> complexity that also causes more problems than it solves: /usr
> partitions.
You are welcome to partition your machines however you please.
Whether a /usr partition is an advantage or a hinderance depends on the
circumstances. Assuming everyone's circumstances will always resemble yours
is a little short sighted, in my opinion. Assuming everyone's _response_ to a
particular situation will also be the same as yours is certainly mistaken.
-Chuck
Chuck Swiger | ch...@codefab.com | All your packets are belong to us.
-------------+-------------------+-----------------------------------
"The human race's favorite method for being in control of the facts
is to ignore them." -Celia Green
Chris Thompson
Rich Teer <ri...@rite-group.com> wrote:
>On 5 Nov 2002, Peter Radcliffe wrote:
>
>> However, if you want to use ufs snapshots on /usr you need it seperate
>> from /
>
>That's not been a problem for me (after stopping xntpd).
Do you think having to stop xntpd is an acceptable operational restiction?
Drazen Kacar
> For that matter, are ELF binaries page aligned on disk, taking up more
> space?
No. If they were, you'd probably be able to change dynamic section and,
oh, add the missing run-path. Although you don't necessarily need page
aligned binary images on disk in order to make that type of adjustment.
Just a file format designed with that feature in mind.
Darren Dunham
news:aq9kmo$g4a$1...@pegasus.csx.cam.ac.uk:
> In article
> <Pine.GSO.4.44.021105...@grover.rite-group.com>, Rich
> Teer <ri...@rite-group.com> wrote:
>>On 5 Nov 2002, Peter Radcliffe wrote:
>>
>>> However, if you want to use ufs snapshots on /usr you need it
>>> seperate from /
>>
>>That's not been a problem for me (after stopping xntpd).
>
> Do you think having to stop xntpd is an acceptable operational
> restiction?
I wonder how hard it would be to get ntp to use /tmp (or whatever) as it's
current directory. Then you shouldn't have a problem with the fssnap.
Or if it could dynamically be moved into the TS group during the slice and
then back to RT afterward.
--
Darren Dunham
ddu...@taos.com
ultra...@hotmail.com
>I don't have any problem with Sun removing static libraries from Solaris so
>long as they keep up the ABI compatibility-- as they have done, and done well,
>in the past.
the SPARC ABI is based on dynamically-linked libraries, you cannot have
an ABI-compliant program that contains system call traps. as the
standard says:
The ABI does not define the implementation of individual system calls.
Instead, programs should use the system libraries that Chapter 6
describes. Programs with embedded system call trap instructions do not
conform to the ABI.
Chuck Swiger
In comp.sys.sun.misc ultra...@hotmail.com wrote:
> In <aq9jqf$clh$1...@shot.codefab.com> "Chuck Swiger" <ch...@codefab.com> writes:
>> I don't have any problem with Sun removing static libraries from Solaris so
>> long as they keep up the ABI compatibility-- as they have done, and done
>> well, in the past.
>
> the SPARC ABI is based on dynamically-linked libraries, you cannot have
> an ABI-compliant program that contains system call traps. as the
> standard says:
[ ... ]
Okay; I'm not sure I see the relevance.
Under Solaris, /bin/true and /bin/false are shell scripts rather than binary
executables, but a minimal assembly implementation of these two programs would
not need to perform any system calls or invoke any library routines at all,
no?
Just store 0x0 or 0xFF into the topmost stack frame, or in the appropriate
register, or however it is that Solaris implements process exit codes.
Casper H.S. Dik
>Under Solaris, /bin/true and /bin/false are shell scripts rather than binary
>executables, but a minimal assembly implementation of these two programs would
>not need to perform any system calls or invoke any library routines at all,
>no?
_exit()?
>Just store 0x0 or 0xFF into the topmost stack frame, or in the appropriate
>register, or however it is that Solaris implements process exit codes.
By calling the _exit() system call.
Chuck Swiger
> "Chuck Swiger" <ch...@codefab.com> writes:
>> Under Solaris, /bin/true and /bin/false are shell scripts rather than binary
>> executables, but a minimal assembly implementation of these two programs
>> would not need to perform any system calls or invoke any library routines at
>> all, no?
>
> _exit()?
A point.
On the other hand, valid one-line C implementations-- int main() { return 0;
}-- do not explicitly call exit() or _exit(). They simply return from
subroutine to a system-provided stub, once called /lib/crt0.o, which was the
thing that passed control to _main() in the first place.
If Solaris will insist that one dynamicly links crt0.o to have the standard
system _exit() symbol available, okay. However, nothing in the code above
requires dynamic linking on systems which provide a static version of _exit.
Roger Marquis
>For a counterexample, look up John Dyson's remarks on the compilation time of
>"make world" under FreeBSD using a staticly linked /bin/sh, make, and compiler
>subsystem, versus that of dynamicly linked system tools.
I tried to replicate those results, on a FreeBSD system, without
success. On my system the static sh was slightly faster in some
cases but even in the best case scenario it was only by 5%, not
nearly enough to justify the hassle of maintaining a separate set
of duplicate binaries.
>> Hope the trend continues with the elimination of another needless
>> complexity that also causes more problems than it solves: /usr
>> partitions.
>
>Whether a /usr partition is an advantage or a hinderance depends on the
>circumstances. Assuming everyone's circumstances will always resemble yours
>is a little short sighted, in my opinion. Assuming everyone's _response_ to a
>particular situation will also be the same as yours is certainly mistaken.
That assumption is yours Chuck. My comment was directed towards
the general case, 99% of /usr partitions, which are A) on the same
disk as /, with B) no different mount options. These legacy
partitions serve no useful purpose and only add a point of failure.
Chuck Swiger
>> For a counterexample, look up John Dyson's remarks on the compilation time
>> of "make world" under FreeBSD using a staticly linked /bin/sh, make, and
>> compiler subsystem, versus that of dynamicly linked system tools.
>
> I tried to replicate those results, on a FreeBSD system, without
> success. On my system the static sh was slightly faster in some
> cases but even in the best case scenario it was only by 5%,
Hmm. Note that I have compared this type of thing in the past, but my
impression is based on a recollection of numbers dating back from the days of
the 25 MHz 68040 and may simply be outdated. I'll try this out against a 4.7
system over the next day or so, and see.
[ And if it turns out that static linking doesn't make much difference, I will
open my mouth even wider and insert my other foot. Deal? :-) ]
> not nearly enough to justify the hassle of maintaining a separate set
> of duplicate binaries.
What hassle? On a stock FreeBSD system:
echo "CFLAGS= -O -pipe -static" > /etc/make.conf
cd /usr/src
make buildworld
>>> Hope the trend continues with the elimination of another needless
>>> complexity that also causes more problems than it solves: /usr
>>> partitions.
>>
>> Whether a /usr partition is an advantage or a hinderance depends on the
>> circumstances. Assuming everyone's circumstances will always resemble
>> yours is a little short sighted, in my opinion. Assuming everyone's
>> _response_ to a particular situation will also be the same as yours is
>> certainly mistaken.
>
> That assumption is yours Chuck.
Which assumption is mine, precisely?
> My comment was directed towards the general case, 99% of /usr partitions,
> which are A) on the same disk as /, with B) no different mount options.
I agree that 99% of /usr partitions are on the same disk as /.
About half of the servers I administer have /usr mounted with different
options than /, since I don't enable softupdates on /. That's a FreeBSD
detail, not a Solaris one, agreed...but also beside the point.
One might want to mount /usr read-only under Solaris, for instance, but I
don't believe that Solaris is especially happy with / being RO.
> These legacy partitions serve no useful purpose and only add a point of
> failure.
You're welcome to your opinion, but you're mistaken if you confuse this
opinion with fact. Specificly, on the system I'm about to test, having /usr
be on a seperate partition from the root partition means that I won't be
changing stuff in /, much less risk filling it, even though "make buildworld"
is going to be placing a few hundred megabytes or so under /usr/obj.
That serves a useful purpose.
But if it simply gave me a warm and fuzzy feeling to have both a / and a /usr
partition, hey, it's not as if you were the one to pay for the hard drive
containing the partition table. Whether a particular reason appears valid to
you (or anyone else not using the system) is not especially relevant.
Casper H.S. Dik
>In comp.unix.solaris Chuck Swiger <ch...@codefab.com> wrote:
>>For a counterexample, look up John Dyson's remarks on the compilation time of
>>"make world" under FreeBSD using a staticly linked /bin/sh, make, and compiler
>>subsystem, versus that of dynamicly linked system tools.
>I tried to replicate those results, on a FreeBSD system, without
>success. On my system the static sh was slightly faster in some
>cases but even in the best case scenario it was only by 5%, not
>nearly enough to justify the hassle of maintaining a separate set
>of duplicate binaries.
"ln /sbin/sh /bin/sh" makes a world of difference for some lmbench micro
benchmarks.
>That assumption is yours Chuck. My comment was directed towards
>the general case, 99% of /usr partitions, which are A) on the same
>disk as /, with B) no different mount options. These legacy
>partitions serve no useful purpose and only add a point of failure.
Hear, hear.
rachel polanskis
Roger Marquis <not-fo...@roble.com> wrote:
> That assumption is yours Chuck. My comment was directed towards
> the general case, 99% of /usr partitions, which are A) on the same
> disk as /, with B) no different mount options. These legacy
> partitions serve no useful purpose and only add a point of failure.
Maybe I am missing something here. I like a separate /usr partition
because of things that have gone wrong with disk drives in the past.
Let's say we have /root and /usr on the same partition slice and
bad blocks start to appear over the area where some critical libraries
reside - what do you do to recover?
If /root and /usr are separate, it is possible to boot single user
or off the CD and add another slice from some other disk, that perhaps
contains the same files as the corrupted slice - you can then
reboot with a new /usr and your back in business, enough to
schedule a more permanent replacement.
Of course, with Jumpstart and Disk mirroring you can have more disaster
recovery options but when I have had to work on systems that have had a
homogenous /root and /usr setup and no other options to rebuild the
system quickly, being able to remove a slice and add in a new one has
been very useful. At least it's been useful for me!
rachel
bit-b...@maney.org
: In article <3dc6d077$0$46609$e4fe...@news.xs4all.nl>,
: Casper H.S. Dik <Caspe...@Sun.COM> wrote:
:>>The only problem may be that you will not be able to replace /usr/lib/libc.so*
:>>if you neither have a static "cp", "mv" nor "tar".
:>
:>Sure you can.
:>
:>"ln" (ln libc.so.1 libc.so.old; ln libc.so.new libc.so.1)
: Correct, I forget about this because it is new to auto remove the ln
: target IIRC
:>"cpio" (will do the rigth thing when overwriting a file;
:> create /usr/lib/XXXXX and rename)
: Mmmm why put effort into a command that has been removed from the POSIX
: standard?
:>and then there's:
:>
:>
:> LD_LIBRARY_PATH=/emergency/lib /emergency/lib/ld.so.1 <cmd> ...
: If Sun supplies this emergency lib it looks like a nice idea.
Umm.... isn't this exactly what the difference is supposed to be in
the legacy /bin, /sbin, /usr/bin, /usr/sbin directory schemes?
Personally I just don't see what getting rid of statically compiled
binaries buys us.
fpsm
--
| Fredrich P. Maney my_last_name AT my_last_name DOT org |
| Do NOT send me HTML formatted E-mail or copies of netnews posts! |
| Address in header is a spamtrap. Use one in signature for replies. |
| Please review http://www.maney.org/uce/ before emailing. |
Casper H.S. Dik
>Personally I just don't see what getting rid of statically compiled
>binaries buys us.
The question is not so much of what it buys you; what does it cost you?
There are certain changes that are impossible to make or really hard to
support by not making this change.
David Williams
"Casper H.S. Dik" <Caspe...@Sun.COM> wrote in message
news:3dce3b4f$0$46598$e4fe...@news.xs4all.nl...
> bit-b...@maney.org writes:
>
> The question is not so much of what it buys you; what does it cost you?
> There are certain changes that are impossible to make or really hard to
> support by not making this change.
>
Rich Teer
> Such as ?
I doubt Casper, or anyone else who knows, could say, especially
at this early stage.
Roger Faulkner
> Casper H.S Dik wrote:
>
> > - partially static linking (libc statically, lib thread not)
> > - applications broke when the libthread/libc interfaces
> > got upgraded
>
> It's been like that for years, so why did it become a problem worth solving
> now? Hmmm... TLS alone can't be the reason... But perhaps you're going to
> make dynamic loading of libthread (and thus making program "threaded" on
> the fly) legal and supported?
I'm not allowed to spill the beans about everything we are doing,
but I must say that you are a good guesser.
Maintaining the static libc, even for Sun's internal use, greatly
restricts what we can do to improve the system.
Roger Faulkner
roger.f...@sun.com
Robert Bonomi
Casper H.S. Dik <Caspe...@Sun.COM> wrote:
>j...@cs.tu-berlin.de (Joerg Schilling) writes:
>
>>In article <usap86h...@corp.supernews.com>,
>>Richard L. Hamilton <rlh...@smart.net> wrote:
>
>>>Seems a copy of libc.so.1 (albeit linked a bit differently?) in /etc/lib
>>>would take care of it. That means root needs to be about 1 MB larger.
>>>If a few more libraries were needed, that might grow to 2 MB larger. OTOH,
>>>you'd get some of that back, inasmuch as dynamically linked binaries are
>>>smaller than statically linked ones.
>
>>The only problem may be that you will not be able to replace /usr/lib/libc.so*
>>if you neither have a static "cp", "mv" nor "tar".
>
>Sure you can.
>
>"ln" (ln libc.so.1 libc.so.old; ln libc.so.new libc.so.1)
>"cpio" (will do the rigth thing when overwriting a file;
> create /usr/lib/XXXXX and rename)
And if neither of -them- are static linked?
>and then there's:
>
>
> LD_LIBRARY_PATH=/emergency/lib /emergency/lib/ld.so.1 <cmd> ...
*BUT*, the _shell_ itself requires the shared libraries _before_ you can
specify the LD_LIBRARY_PATH.
The disadvantage of static linking _is_ that you have to send out bigger
'patchfiles' when something that affects a static-linked binary changes.
The disadvantage of dynamic linking is that if the dynamic library gets
clobbered *NOTHING* works.
In the case of file-system corruption, static-linked executables who's
filesystem entry (singular) is intact *still*work*. In the case of a
dynamically-linked executable, the executable's entry must be intact
*AND* those for _every_ shared library it links against.
The "opportunity" for failure is *MUCH* greater.
In the case of _file_ damage, the same thing applies. If the damage is
to a static-linked executable *only* THAT executable is affected. If
the damage is to a dynamic-link library, *everything* that relies on that
library stops working.
FORCING everything to use dynamic linking *greatly* increases the number
of dependancies (other things that must _also_ be working properly) for
_minimal_ system operation. *BAD*MOVE* from a _reliability_ standpoint.
It adds a whole sh*tload of additional "single point of failure" modes.
One will need to keep a 'back up root partition', including the critical
"/usr/lib/*" shared libraries.
And develop a 'multiple path' boot loader. So that if booting from the
'master root' partition fails, one automatically proceeds to reboot using
th 'back-up root partition'. with automatic scripting to 'repair' the
'master root' partition, and then reboot yet again (back to the 'master
root' partition).
Hell, maybe you *always* boot from the 'back-up root' partition -- which
"sanity-checks" the master partition, fixes any problems there, and reboots
off the master. Assuming the 'backup root' runs RO, this _should_ be safe.
I *HOPE* Sun will provide a 'master list' of what all the "critical files"
are, i.e. *EVERYTHING* that -HAS- to be present to get the system up and
running in _single-user_ mode. If you can't get as far as a single-user
shell prompt, you can't do *anything* towards fixing what's wrong.
Robert Bonomi
Casper H.S. Dik <Caspe...@Sun.COM> wrote:
>
>>Personally I just don't see what getting rid of statically compiled
>>binaries buys us.
>
>The question is not so much of what it buys you; what does it cost you?
What it costs is "reliability".
Shared libraries introduce "yet another" a 'single point of failure' with
catastrophic consequences.
>There are certain changes that are impossible to make or really hard to
>support by not making this change.
They are not 'impossible', nor 'really hard' to make. The correct word is
"inconvenient". As in, "we would have to distribute new copies of the
affected static-linked executables in addition to the library itself, and
customers would have to know to re-link any static-linked ex"utables of
their own. Which means that we would have to _tell_ them that the world
changed."
Relying *exclusively* on dynamic-link makes a number of things "convenient"
for the O/S and compiler-tools people. *AT*THE*EXPENSE*OF*THE*CUSTOMER*.
The production environment is now more 'fragile'. *MORE* things have to
'go right' for *anything* to work. More things have to *be* right, in order
to fix problems *WITHOUT* having to resort to booting from distribution
media.
*FORCING* people to do things in -one- way, where there were alternatives,
implies that you (Sun, collectively, not you, personally, Casper:) are
convinced you know better than your customers, as far as what is 'right'
for *their* shop. That kind of arrogance was historically the realm of
IBM, and Microsoft.
Casper H.S. Dik
>What it costs is "reliability".
Reliability is the realm of statistics.
My kernel has currently 148 modules loaded; I'm sure that if even a
small percentage of those failed to load the system would not boot.
You're worried about 2 or 3 shared libraries. But you're ignoring
the 100-odd files that already need to be successfully loaded before
you even get to that point.
Some of you seem to be assuming that those shared library are in /usr/lib.
Of course they cannot be.
>Relying *exclusively* on dynamic-link makes a number of things "convenient"
>for the O/S and compiler-tools people. *AT*THE*EXPENSE*OF*THE*CUSTOMER*.
>The production environment is now more 'fragile'. *MORE* things have to
>'go right' for *anything* to work. More things have to *be* right, in order
>to fix problems *WITHOUT* having to resort to booting from distribution
>media.
Quantify "expense". What does it cost you to not have static
libraries? How many failure do you remember where you didn't have
the lirbaries needed to run the shell (and not having /usr doesn't count;
assume the libraries they're not in /usr)?
You have no idea how fragile the current situtation of partially
statically linked executable linked against libc, then loading libc.so
when they need anything else, but all libc.so entry points being resolved
agaisnt the original static executable really is because there are
so many differences between the static libraries and the dynamic libraries.
There's a lot of stuff that is not and cannot be done in static libraries;
the static libraries are their but they are *different* from the
dynamic ones in some subtle ways. There are things we cannot do;
there are bugs that arise because we cannot do thes things.
>*FORCING* people to do things in -one- way, where there were alternatives,
>implies that you (Sun, collectively, not you, personally, Casper:) are
>convinced you know better than your customers, as far as what is 'right'
>for *their* shop. That kind of arrogance was historically the realm of
>IBM, and Microsoft.
I cannot be swayed by the "expense" argument.
You're almost saying as it it's two executables for single user
(/sbin/init and /sbin/sh) and that the additional risk is /lib/*
(really two or three additional files)
But in actual fact, to just run these programs you need to have:
/sbin/init
/etc/lib/ld.so.1
/etc/lib/libdl.so.1
/etc/inittab
- doesn't run when /var isn't mounted
- autopush
- soconfig
/etc/rcS
/sbin/sh
and a cast of 100 kernel modules and .conf files.
What's added in the mix is probably something in the order of:
/lib/ld.so.1 (trades against /etc/lib/ld.so.1)
/lib/libc.so.1
/lib/libdl.so.1 (trades against /etc/lib/libdl.so.1)
/lib/libgen.so.1 (also needed for /bin/sh)
(I'm not familiar with the particulars of the project so these pathnames
might be wrong)
So to a cast of around a hundred we're adding *two* files to get to
single user.
Is that really such a big decrease in reliability that it warrants the
use of the caps lock key?
I can understand why it makes you feel uneasy about not having the "comfy"
static libraries. But if I look at the cold hard fact of "single user boot
failure root causes" plus the above tabulation of additional files needed
as percentage of the total needed on boot, I'm much less worried.
Drazen Kacar
> You have no idea how fragile the current situtation of partially
> statically linked executable linked against libc, then loading libc.so
> when they need anything else, but all libc.so entry points being resolved
Some libc.so entry ponits, which makes matters worse. You typically won't
have the whole libc archive linked into the static executable. But there
are ways to deal with this, as long as the ownership of %g7 doesn't
matter.
> agaisnt the original static executable really is because there are
> There's a lot of stuff that is not and cannot be done in static libraries;
I wonder if those things can be done with AIX's XCOFF objects.
> What's added in the mix is probably something in the order of:
>
> /lib/ld.so.1 (trades against /etc/lib/ld.so.1)
> /lib/libc.so.1
> /lib/libdl.so.1 (trades against /etc/lib/libdl.so.1)
> /lib/libgen.so.1 (also needed for /bin/sh)
/bin/sh has become a monster these days. Granted, most of the dependencies
are designated for lazy loading[1], but they will be loaded under some
circumstances. Currently "ldd /bin/sh" on Solaris 9 gives:
libgen.so.1 => /usr/lib/libgen.so.1
libsecdb.so.1 => /usr/lib/libsecdb.so.1
libnsl.so.1 => /usr/lib/libnsl.so.1
libc.so.1 => /usr/lib/libc.so.1
libcmd.so.1 => /usr/lib/libcmd.so.1
libdl.so.1 => /usr/lib/libdl.so.1
libmp.so.2 => /usr/lib/libmp.so.2
libc_psr filter
/bin/zsh has a smaller list of dependencies, therefore... :-)
[1] libc is supposed to be lazy loaded (although that has no effect
whatsoever, except the possibility of triggering bugs), but libgen
isn't. Curious.
Richard L. Hamilton
bonomi@c-ns. (Robert Bonomi) writes:
> In article <3dc6d077$0$46609$e4fe...@news.xs4all.nl>,
> Casper H.S. Dik <Caspe...@Sun.COM> wrote:
>>j...@cs.tu-berlin.de (Joerg Schilling) writes:
>>
>>>In article <usap86h...@corp.supernews.com>,
>>>Richard L. Hamilton <rlh...@smart.net> wrote:
>>
>>>>Seems a copy of libc.so.1 (albeit linked a bit differently?) in /etc/lib
>>>>would take care of it. That means root needs to be about 1 MB larger.
>>>>If a few more libraries were needed, that might grow to 2 MB larger. OTOH,
>>>>you'd get some of that back, inasmuch as dynamically linked binaries are
>>>>smaller than statically linked ones.
>>
>>>The only problem may be that you will not be able to replace /usr/lib/libc.so*
>>>if you neither have a static "cp", "mv" nor "tar".
>>
>>Sure you can.
>>
>>"ln" (ln libc.so.1 libc.so.old; ln libc.so.new libc.so.1)
>>"cpio" (will do the rigth thing when overwriting a file;
>> create /usr/lib/XXXXX and rename)
>
> And if neither of -them- are static linked?
As long as both old and new versions of libc.so.1 are usable, the
above would work even with dynamically linked versions. If your point
is that neither they nor anything else dynamically linked would work if
their dynamic dependencies weren't usable, that's clear enough.
One could have the few commands that are now statically linked instead be
dynamically linked but with a run path of /etc/lib instead of /usr/lib,
and all required dependencies duplicated in /etc/lib. That would be
almost as good as the regular arrangement, and the small set of libs
required probably wouldn't require more than a couple of MB extra disk
space. The one sense in which it wouldn't be quite as good is that
there's more likelihood of metadata damage with the larger number of
total files involved (but less of data damage, since library code won't
be duplicated in all the alternatively (rather than statically) linked
files; and one could argue that data damage is scarier than metadata
damage, since data damage would be more likely to go undiagnosed, unless
you used ASET or tripwire to regularly check the critical files).
--
mailto:rlh...@mindwarp.smart.net http://www.smart.net/~rlhamil
Tim Bradshaw
> I can understand why it makes you feel uneasy about not having the "comfy"
> static libraries. But if I look at the cold hard fact of "single user boot
> failure root causes" plus the above tabulation of additional files needed
> as percentage of the total needed on boot, I'm much less worried.
Actually, just how hard would it be, technically, to write something
which took a shared library, and an executable that uses it, and
`staticized' it? This is pretty much what the shared linker does
anyway, at least conceptually. Obviously the code in the now-static
binary would be largely relocatable, so you'd lose some tiny bit of
performance, but that's not the issue the static-library wanters are
after, I think. One could even invent some kind of executable which
essentially was the original executable + all the libraries, + a
runtime linker which would glue them all together. The executables
would be kind of big, I guess...
This clearly isn't *trivial*, but I'm not sure how hard it would be.
If it existed then you could ship a shared-only system with a tool
that let people who cared construct static executables...
--tim
Drazen Kacar
> One could even invent some kind of executable which essentially was the
> original executable + all the libraries, + a runtime linker which would
> glue them all together. The executables would be kind of big, I
> guess...
What follows is a truly sick idea.
Remove the "no directories" limitation from extended file attributes.
Create a "lib" directory there and put all the libraries under it. Extend
dynamic loader with $EFATTRS token and link the executable with
"$EFATTRS/lib" in runpath. Extend ELF to allow interpreter to reside
there, as well.
Isn't it marvelous?
Tim Bradshaw
> Remove the "no directories" limitation from extended file attributes.
> Create a "lib" directory there and put all the libraries under it. Extend
> dynamic loader with $EFATTRS token and link the executable with
> "$EFATTRS/lib" in runpath. Extend ELF to allow interpreter to reside
> there, as well.
> Isn't it marvelous?
Well, it's kind of reminiscent of how the Mac does it, I think
(resource forks?). I meant something much more mundane, but this is
kind of a cool idea, though it makes me feel pretty queasy...
--tim
Drazen Kacar
> Drazen Kacar <da...@willfork.com> writes:
> > What follows is a truly sick idea.
>
> > Remove the "no directories" limitation from extended file attributes.
> > Create a "lib" directory there and put all the libraries under it. Extend
> > dynamic loader with $EFATTRS token and link the executable with
> > "$EFATTRS/lib" in runpath. Extend ELF to allow interpreter to reside
> > there, as well.
>
> for open()/stat()/etc that would allow to use attributes
> as context depended files
What do you want to use them for? C++ libraries are inviting. Naah, resist
the temptation. Magick always backfires.
Chuck Swiger
> Actually, just how hard would it be, technically, to write something
> which took a shared library, and an executable that uses it, and
> `staticized' it? This is pretty much what the shared linker does
> anyway, at least conceptually.
MacOS X does something very close to this now, in order to reduce the dynamic
linking costs by 'prebinding' references to dynamic symbols. From 'man ld':
-prebind
Have the static linker, ld(1), prebind an exe-
cutable's or dynamic shared library's undefined
symbols to the addresses of the dynamic libraries
it is being linked with. This optimization can
only be done if the libraries don't overlap and no
symbols are overridden. When the resulting program
is run and the same libraries are used to run the
program as when the program was linked, the dynamic
linker can use the prebound addresses. If not, the
dynamic linker undoes the prebinding and binds nor-
mally. This option can also be specified by set-
ting the environment variable LD_PREBIND. If the
environment variable LD_FORCE_NO_PREBIND is set
both the option -prebind LD_PREBIND environment
variable are ignore and the output is not prebound.
> Obviously the code in the now-static binary would be largely relocatable, so
> you'd lose some tiny bit of performance, but that's not the issue the
> static-library wanters are after, I think.
Whether position-independant code performs more slowly, or the extent to which
is does, is both architecture- and CPU-implementation dependant.
> One could even invent some kind of executable which essentially was the
> original executable + all the libraries, + a runtime linker which would glue
> them all together. The executables would be kind of big, I guess...
If you're producing a staticly linked executable, you don't need all of the
libraries or a runtime linker...that's kinda the point of static linking!
When you staticly link, you only include the symbols which are being
referenced, which means that "Hello, World" needs to load in puts() and
whatever that depends on from stdlib, but it won't link in
printf()/vsprintf(), unused math routines like cos(), and everything else that
isn't needed.
Drazen Kacar
> In comp.sys.sun.misc Tim Bradshaw <t...@cley.com> wrote:
> > Actually, just how hard would it be, technically, to write something
> > which took a shared library, and an executable that uses it, and
> > `staticized' it? This is pretty much what the shared linker does
> > anyway, at least conceptually.
>
> MacOS X does something very close to this now, in order to reduce the dynamic
> linking costs by 'prebinding' references to dynamic symbols. From 'man ld':
>
> -prebind
Also known as QuickStart on IRIX and Tru64 and prelinking on Linux. But
none of those methods create a static binary.
Jonathan Biggar
> > Actually, just how hard would it be, technically, to write something
> > which took a shared library, and an executable that uses it, and
> > `staticized' it? This is pretty much what the shared linker does
> > anyway, at least conceptually.
>
> MacOS X does something very close to this now, in order to reduce the dynamic
> linking costs by 'prebinding' references to dynamic symbols. From 'man ld':
>
> -prebind
So is that what a MacOS X installer is doing when it is in the
"optimizing" phase?
Inquiring minds want to know.
--
Jon Biggar
Floorboard Software
j...@floorboard.com
j...@biggar.org
Joerg Schilling
Chuck Swiger <ch...@codefab.com> wrote:
>In comp.sys.sun.misc Tim Bradshaw <t...@cley.com> wrote:
>> Actually, just how hard would it be, technically, to write something
>> which took a shared library, and an executable that uses it, and
>> `staticized' it? This is pretty much what the shared linker does
>> anyway, at least conceptually.
>
>MacOS X does something very close to this now, in order to reduce the dynamic
>linking costs by 'prebinding' references to dynamic symbols. From 'man ld':
>
> -prebind
> Have the static linker, ld(1), prebind an exe-
> cutable's or dynamic shared library's undefined
> symbols to the addresses of the dynamic libraries
Please keep in mind that ld(1) on MacOS X is not standard compliant.
It ignores library search rules (you cannot e.g re-define malloc()).
This may be aproblem caused by the fact that MacOS X (the same
as it's precursor Next Steo uses a dynamic linking scheme derived from
ancient SySVr3 linking.
--
EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
j...@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schi...@fokus.gmd.de (work) chars I am J"org Schilling
URL: http://www.fokus.gmd.de/usr/schilling ftp://ftp.fokus.gmd.de/pub/unix
Drazen Kacar
> In article <ar8l0m$18sa$1...@shot.codefab.com>,
> Chuck Swiger <ch...@codefab.com> wrote:
> >MacOS X does something very close to this now, in order to reduce the dynamic
> >linking costs by 'prebinding' references to dynamic symbols. From 'man ld':
> >
> > -prebind
> > Have the static linker, ld(1), prebind an exe-
> > cutable's or dynamic shared library's undefined
> > symbols to the addresses of the dynamic libraries
>
> Please keep in mind that ld(1) on MacOS X is not standard compliant.
Which standard do you have in mind?
> It ignores library search rules (you cannot e.g re-define malloc()).
--