Binaries considered harmful
Paul Allen
I have a proposal that I'd like people to consider.
I've been following Linux from the sidelines since last year sometime.
I've noticed that with each release some of the old binaries from the
previous release no longer work. When people ask why some command or
other no longer works, the answer is often to recompile with the new
library, or with the new include files. It seems to me that having
lots of old binaries around on the archive sites is causing more
problems than it's worth. Clearly, you need to have binaries for the
compiler and enough commands to bootstrap a system. I think it's
counterproductive in the long run to distribute binaries for commands
that are not essential for bootstrapping. They'll all eventually
be broken by some new release.
So here's my proposal:
- In addition to the binary boot and root images that currently
accompany new releases, a companion tarfile should be released
containing things like the compiler for that release and other
stuff that was too large to fit in the root image.
- Software that is not needed for bootstrapping should be placed in the
archives in source or diff form only.
- Part of the process of installing a new Linux release is to recompile
everything.
I may be missing something really obvious here, but I just don't see
the need for all these binaries. I'd rather just have sources. With
the exception of a few things that simply couldn't be compiled, Minix
has always been a source-only system. (You got the binaries when you
bought Minix, and sources/diffs from the net kept you current from then
on.) I see no reason that Linux needs to work differently.
I invite your comments, criticisms, flames, or whatever.
Paul Allen
pal...@atc.boeing.com
Hongjiu Lu -- Graduate Student
My proposal is use .a instead of binary to distribute most of programs.
The .a is more stable than binary. When a new library is out, you just
need to
gcc -o foo foo.a -lneededbyfoo
Take a look at 2.1shared-A.tar.Z. They are all .a's. You can link them
against any libraries, shared, static or update.
Of course, you need gcc and ld.
H.J.
Drew Eckhardt
>
>I have a proposal that I'd like people to consider.
>
>I've been following Linux from the sidelines since last year sometime.
>I've noticed that with each release some of the old binaries from the
>previous release no longer work. When people ask why some command or
>other no longer works, the answer is often to recompile with the new
>library, or with the new include files. It seems to me that having
>lots of old binaries around on the archive sites is causing more
>problems than it's worth. Clearly, you need to have binaries for the
>compiler and enough commands to bootstrap a system. I think it's
>counterproductive in the long run to distribute binaries for commands
>that are not essential for bootstrapping. They'll all eventually
>be broken by some new release.
>
Other than programs which depend on the internal structure of
Linux (this would be the ps packages, the file system utilities
such as fdisk, etc), what programs have broken with new releases of
Linux?
Some programs suffered because of bad library functions,
and needed recompiling, but there's a difference
between one person recompiling at several hundred.
Except for files I have corrupted, or that were broken
in the first place and subsequently fixed, much of what
I'm using is from the .11 distribution (the file
utilities, bash until just recently, awk, sed, make, etc). I
had been compiling my kernel with the original 'C' compiler
until 2.1 offered C++ and shared libaries, sufficient
incentive to switch.
>So here's my proposal:
>
>- In addition to the binary boot and root images that currently
> accompany new releases, a companion tarfile should be released
> containing things like the compiler for that release and other
> stuff that was too large to fit in the root image.
The difficulty with one big tar file is in downloading it,
passing it through your internet site without incuring the wrath
of the sysadmins and disk quotas, transfering it on floppy
disks, etc.
>- Software that is not needed for bootstrapping should be placed in the
> archives in source or diff form only.
I'm glad you have the disk space and time to spare. I don't.
The last time I built gcc, it used over 30M of disk space. The last time
I built X, it took close to 200M of disk space, and 24 hours to
build on a 50Mhz 68030 with 16M of real memory (This is somewhat longer
than it had to be, as the
disks were NFS mounted). X is an extreme case, but exemplifies the
kind of software that people will be running on Linux as it becomes
more refined, stable, and popular.
The smaller programs (shell utils, file utils, mtools, elvis / micro
emacs) can be dealt with, but it is still more convienient for users
to have a binary available so they don't have to do it themselves.
>- Part of the process of installing a new Linux release is to recompile
> everything.
>
When I've installed a new linux release, it's
consisted of getting the new boot image, getting the new kernel
source, and performing the following procedure :
cd /usr/tmp
tar xvf /dev/b1.44
zcat /usr/tmp/bootimage* > /vmunix+
Reboot vmunix+. If it works, untar the sources,
add in my favorite patches, do a make (About 8
minutes with the old C compiler on a 386-33), and boot
/usr/src/linux/Image.
If that works
mv /vmunix /vmunix-
mv /usr/src/linux/Image /vmunix
The only thing this has ever broken was the simple ps package.
>I may be missing something really obvious here, but I just don't see
>the need for all these binaries. I'd rather just have sources. With
>the exception of a few things that simply couldn't be compiled, Minix
>has always been a source-only system. (You got the binaries when you
>bought Minix, and sources/diffs from the net kept you current from then
>on.) I see no reason that Linux needs to work differently.
>
>I invite your comments, criticisms, flames, or whatever.
>
In conclusion :
1. Things are not as unstable as you claim they are, and I would
venture to say that most upgrades are in pursuit of
having the latest, neat new features, or bug fixes to
the source.
2. Binary distributions are significantly smaller than
source distributions, take less time to
download, less disk space, and less time
because they don't need to be compiled.
An example of a mid size program, gdb, in source and
binary distributions. The binary is ~800k uncompressed,
the source 8M, ten times as large.
-rw-r--r-- 1 drew wheel 3105370 Apr 2 01:24 gdb-4.4.tar.Z
-rw-r--r-- 1 drew linux 469777 Mar 11 02:06 gdb-4.4.tar.Z
3. Linux is getting real users, not hackers. These people
want something that works like DOS, they
see something they like, grab it from their
favorite BBS, and run it. Period.
I'm not saying that it's a good thing,
but given the little trouble it takes,
it's not a bad thing to offer a binary
distribution for these people.
Source should be freely available to anyone who wants it,
but a more convienient binary distribution should also
be available.
Greg Lee
}...
}Source should be freely available to anyone who wants it,
}but a more convienient binary distribution should also
}be available.
Sounds good to me. But there has never been source code available for any
of the versions of gcc, so far as I know, nor diffs against the fsf distribution.
I hope there will be. I have never found source code for update. The source
code I have for fsck doesn't come close to compiling with the current linux
include files. I'm going to work on fsck.c (I question whether fsck -m works
properly now), but since I don't understand the filesystem, my prospects aren't
very good.
Anybody with any source, please upload.
--
Greg Lee <l...@uhunix.uhcc.hawaii.edu>
Daniel D Deavours
>- Software that is not needed for bootstrapping should be placed in the
> archives in source or diff form only.
>- Part of the process of installing a new Linux release is to recompile
> everything.
There's a lot of good reasonds for distributing binaries. Emacs
18.58 is a good reason! The source code is 4 megabytes compressed, and
the time it would take to compile it on a 16mH SX with 2 megs of ram and
a slow hard drive would be measured in days. Don't even think about X.
>I may be missing something really obvious here, but I just don't see
>the need for all these binaries. I'd rather just have sources. With
>the exception of a few things that simply couldn't be compiled, Minix
>has always been a source-only system. (You got the binaries when you
>bought Minix, and sources/diffs from the net kept you current from then
>on.) I see no reason that Linux needs to work differently.
Minix is mostly a 16 bit OS, and has silly silly size limits to
everything. Linux is different because it can compile and run HUGE programs.
(I easily ported gnuchess, try that on Minix.) Also, keeping the source
code around for all the utilities you need can be pretty hard on a limited
sized hard drive. Further, what compiles on gcc1.37 may not compile on
1.40 or 2.0 or 2.1 (it should, but....) It's just so much more convenient.
I like the idea, however, of keeping around sources and diffs.
Shared libraries are one of them. I think that keeping binaries, even though
some day they may break, are a good idea also.
>Paul Allen
>pal...@atc.boeing.com
Dan Deavours
ze...@uiuc.edu
Will Clark
>
>I have a proposal that I'd like people to consider.
>
>- In addition to the binary boot and root images that currently
> accompany new releases, a companion tarfile should be released
> containing things like the compiler for that release and other
> stuff that was too large to fit in the root image.
That's one hell of a tar! I'm sure plenty of people are eager to
download this via modem. :-) (The phones is this building can't get
through more than about 100 K before trashing the connection. We *do*
have ethernet-to-internet too, but not everyone does.)
>- Software that is not needed for bootstrapping should be placed in the
> archives in source or diff form only.
>
>- Part of the process of installing a new Linux release is to recompile
> everything.
Aieeee! Nooooo! Don't get rid of the binaries. I'm not good enough
with C, patching, etc. to recompile every time a new version comes
out. If this happens, I'm really screwed. Yes this is a hacker's
kernel/system, but there are some people who'd like to learn about
*nix with a good free system, (I am certainly one of those) but are
not necessarily good at programming. (I'm one of these, too.)
>I may be missing something really obvious here, but I just don't see
>the need for all these binaries. I'd rather just have sources. With
>the exception of a few things that simply couldn't be compiled, Minix
>has always been a source-only system. (You got the binaries when you
>bought Minix, and sources/diffs from the net kept you current from then
>on.) I see no reason that Linux needs to work differently.
If you're worried about binaries not running under the newest release,
wait a couple of weeks/months before moving over. The stuff that
needs to be hacked and recompiled will be, and the rest doesn't need
changing. If you need the new release, and can fix stuff yourself, do
so. Happily, there are plenty of these people out there who upload
their ports for FTP, which means *I* can play with this great OS!
>I invite your comments, criticisms, flames, or whatever.
Always glad to help!
>Paul Allen
>pal...@atc.boeing.com
Will Clark (Hoping the Giants do a little better this year.)
wil...@site007.saic.com
------------------------
SAIC does not share my opinions.
harlick@lindy is a friend's account, but you can send me email here, too.
Jim Winstead Jr.
>I have never found source code for update. The source code I have for
>fsck doesn't come close to compiling with the current linux include
>files.
I have the (as I know it) most current source for all of this, and
I'll be uploading them as a package today (hopefully) to tsx-11, etc,
and they should be available soon. Look for system-0.95a.tar.Z.
Included:
chsh - new to this release (change shell for a user)
doshell - start a process on another tty
fdisk-x - fdisk that understands extended partitions
fdisk - the old fdisk
hostname- set your computer's name!
mkfs - make a filesystem
mknod - make a device
mkswap - make a swap device/file
mount - mount a filesystem
passwd - change a user's password
pwd - print working directory (from BSD)
rootdev - display/change root device (rdev.c)
setterm - set terminal characteristics
swapon - use a swap file/partition
sync - sync the filesystem (monster source! :)
umount - unmount a filesystem
update - background update thing - syncs filesystem
Like I said, keep an eye out for 'em - they are mostly the versions
from the 0.95a root disk, with some cleanup of the code, the addition
of a couple of minor utilities, and some other minor changes.
Er, now that I look at the list again, I notice I don't have fsck.
If I remember correctly, Linus mentioned having lost this source in a
hard drive disaster - anyone want to rebuild it?
--
Jim Winstead Jr. (CSci '95) | "Catch a fish!"
Harvey Mudd College | -Geddy Lee,
jwin...@jarthur.Claremont.EDU | San Diego Sports Arena
Disclaimer: Mine, not theirs! | January 20, 1992
Kevin Cummings
Not all of these executables are on the 0.95a root disk. fdisk-x,
hostname, and passwd. Who is packaging these executables for us?
(I downloaded hostname from tsx-11, and got passwd from my 0.95
root disk. Where can I find fdisk-x?????)
=================================================================
Kevin J. Cummings Prime Computer Inc.
20 Briarwood Road 500 Old Connecticut Path
Framingham, Mass. Framingham, Mass.
InterNet: cumm...@primerd.Prime.COM
UUCP: uunet!primerd.Prime.COM!cummings
Std. Disclaimer: "Mr. McKittrick, after careful consideration,
I've come to the conclusion that your new
defense system SUCKS..." -- War Games
=================================================================
Eps...@dockmaster.ncsc.mil
people's systems (-: and lots of superusers use vi vice emacs when
logged in as root need to watch out for things like hlu's latest GCC ---
there is a .exrc in 2.1ps.tar.Z This caused grief when I did my first
download via kermit under Linux as I lose/garbage 1 out of the last 9
bytes of a binary file [FEATURE of the download process thru a 36 bit
machine] The uncompress and tar left a bad .exrc file in the ps
directory There should be no reason to be including .exrc files in the
GCC
John
Linus Benedict Torvalds
>of the versions of gcc, so far as I know, nor diffs against the fsf distribution.
Guilty as charged: I made a small gcc.cdiff available for the original
gcc-1.40 release, but I don't think I made it "public" - I just probably
mailed it to some interested parties.
Btw, that cdiff wasn't big, and I think it was the biggest one to date:
the original gcc-1.40 for linux contained code to fix the broken
-msoft-float (mostly thanks to Bruce Evans), as well as my small hack to
use the 386 string instructions (2.x does it, but 1.40 didn't). The
newer versions of gcc for linux are probably almost exactly the fsf
distribution: the problems have mostly been the libraries.
>I have never found source code for update.
This one is mine as well: it's about 10 lines of code, but I never made
that one public either. That too has been sent out (or rather:
recreated on the fly) to interested parties. If anybody is interested,
update.c looks something like this:
#include <unistd.h>
#include <signal.h>
void alrm_fn(int sig)
{
sync();
signal(SIGALRM,alrm_fn);
alarm(30);
}
main()
{
alrm_fn(SIGARLM);
while (1)
pause();
}
I won't guarantee that one compiles: I just wrote it in, but it will
give you an idea of the thing.
>The source code I have for fsck doesn't come close to compiling with
>the current linux include files.
My dog ate it... Well, close: I had (really!) a new version of fsck.c
that had some minor fixes, as well as compiling with the new
include-files, but I hadn't released it yet - I was still working on the
VFS code. Then I made a minor mistake in my kernel code, and oops..
linux started eating up my filesystem. I didn't loose anything vital
/except/ the new fsck.c. The binary was intact, but the sources
weren't. So the answer is that there simply /isn't/ any fsck code
available which compiles with the new header files unless somebody else
has written one..
> I'm going to work on fsck.c (I
>question whether fsck -m works properly now), but since I don't
>understand the filesystem, my prospects aren't very good.
I know the filesystem (boy do I know it!), and I've intended to recreate
my sources for a long time, but something has always come up. I won't
promise I'll do it anytime soon, so if anybody else feels like doing it
(starting from the old sources), it would be nice..
Linus
Rogier Wolff
>In article <1992Apr7.0...@athena.mit.edu> pa...@atc.boeing.com writes:
>>(Subject line with appologies to E. Djikstra. :-))
>>
Everybody seems to disagree, however I think that some of the archives need to
be cleaned up: I downloaded Linux 0.95 yesterday: Total of 15Mb, It just fits
on 6 floppies if you delete all the binaries that are doubled (less than 10M)!
I think that someone should take a freshly installed system, and make nice
uniform tar files for everything that is there.
Roger
--
If the opposite of "pro" is "con", what is the opposite of "progress"?
(stolen from kadokev@iitvax ==? tech...@iitmax.iit.edu)
EMail: wo...@duteca.et.tudelft.nl ** Tel +31-15-783644 or +31-15-142371
Jim Winstead Jr.
>In article <1992Apr7.2...@muddcs.claremont.edu>, jwin...@jarthur.claremont.edu (Jim Winstead Jr.) writes:
>> I have the (as I know it) most current source for all of this, and
>> I'll be uploading them as a package today (hopefully) to tsx-11, etc,
>> and they should be available soon. Look for system-0.95a.tar.Z.
>>
>> Included:
>> chsh - new to this release (change shell for a user)
>>
>> from the 0.95a root disk, with some cleanup of the code, the addition
>> of a couple of minor utilities, and some other minor changes.
>
>Not all of these executables are on the 0.95a root disk. fdisk-x,
>hostname, and passwd. Who is packaging these executables for us?
Right - I thought I made it clear that there was the addition of a
couple of minor utilities, such as hostname, passwd, chsh, etc.
fdisk-x is actually just a new version of fdisk, but I have kept the
old fdisk around for now. Eventually, fdisk-x will replace fdisk.
As far as I know, nobody is packaging the executables - I'm releasing
the source, but not executables, since they can easily be compiled.
Once the ABC Release is out, this sort of issue will be moot - I
expect both binaries and sources for this to be part of that. Until
then, you'll have to recompile them yourself, or find someone else to
do it.
>(I downloaded hostname from tsx-11, and got passwd from my 0.95
>root disk. Where can I find fdisk-x?????)
The source can be found on tsx-11 in
/pub/linux/patches/fdisk-extended.c, or something very similar. It
will also be available in the package I'm putting together. The
executable, as of yet, is unavailable.
A quick, fairly uncertain word on a v0.95c root disk. Don't count on
seeing one! The 0.95a root disk will still work, and has no
unsurmountable problems. I would much rather wait for the 0.96
release of Linux to do another root disk. I may change my mind,
however, so keep your ears open.
Joe Smith
I'll probably regret this, but here goes...
What's the beef? I think the discussion about distributing individual
binaries is clouding a more important issue. It seems pretty clear to
me that linux, just like any Unix, needs a binary distribution to
bootstrap from. The binary distribution needs to be
* stable
* easily downloaded & installed
* complete(*)
* as small as practical (say ~5-8 Mb)
This would allow newcomers to install a working system with known
features (& bugs) with a minimum of hassle and ftp/file transfer time.
I'd like to see this system include gcc(**), but that's certainly not
necessary. From this platform, people could evaluate linux without
spending a couple of weeks moving this file and that patch, etc. If
successful, they can then add binary accessory packages or source +
patches. The distribution model used by commercial Unixes is not bad
- it just costs way too much, and you can't get source even if you
want it.
I don't know if Alan's ABC version will do this or not, but I hope
so. Without a stable C compiler & development system it's probably a
waste of time to try and assemble a distribution, but now I think we
can.
<Joe
------------------------------
(*) What's complete? Well, one example comes to mind: the MKS toolkit
is reasonably complete and fits into this size limitation. Basically
it should have enough functionality to give a good look at Unix and
the capability to load additional packages. Maybe the Byte benchmarks
would be a reasonable target capability.
(**) What would you do with such a system except software development:
I'm not holding *my* breath for Word Perfect for Linux :-)
--
Joe Smith
University of Pennsylvania j...@mbio.med.upenn.edu
Dept. of Biochemistry and Biophysics (215) 898-8348
Philadelphia, PA 19104-6059