klibc only initramfs
Goswin von Brederlow
I googled a bit and found this old mail about a klibc only initramfs:
http://lists.debian.org/debian-kernel/2006/07/msg00400.html
I would really like to do this and it has been close to 4 years since
that mail. But it doesn't look like there has been much progress or not
in the right direction. Looking at my initramfs I see:
% ls lib
cryptsetup/ libm.so.6
klibc-zUXi_KjK5ZQAIyc8jlwme9T6a4U.so* libncurses.so.5
ld-linux.so.2* libpopt.so.0
libc.so.6* libreadline.so.5
libcfont.so.0 libselinux.so.1
libconsole.so.0 libuuid.so.1
libctutils.so.0 libvolume_id.so.0
libdevmapper.so.1.02.1 modules/
libdl.so.2 udev/
% ls bin
busybox* dmesg* insmod* minips* nfsmount* reboot* sleep* zcat*
cat* false* ipconfig* mkdir* nuke* resume* sync*
chroot* fstype* kill* mkfifo* pivot_root* run-init* true*
cpio* gunzip* ln* mknod* poweroff* sh* umount*
dd* halt* loadkeys* mount* readlink* sh.shared* uname*
So, while I can build a trivial initramfs with klibc only, as soon as I
want md, lvm or crypt I will be pulling in libc6 and a bunch of other
libs as well as busybox. Exactly those things the klibc should replace.
Could we build stripped down versions of those tools (and libs as
required) build against klibc? I certainly see no need for ncurses in my
initramfs. Building a klibc based initramfs that then includes libc6
(and even busybox) as well seems rather stupid. One without klibc would
be smaller then.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87zl34i...@frosties.localdomain
Michael Tokarev
> Hi,
>
> I googled a bit and found this old mail about a klibc only initramfs:
>
> http://lists.debian.org/debian-kernel/2006/07/msg00400.html
>
> I would really like to do this and it has been close to 4 years since
> that mail. But it doesn't look like there has been much progress or not
> in the right direction. Looking at my initramfs I see:
>
> % ls lib
> cryptsetup/ libm.so.6
> klibc-zUXi_KjK5ZQAIyc8jlwme9T6a4U.so* libncurses.so.5
> ld-linux.so.2* libpopt.so.0
> libc.so.6* libreadline.so.5
> libcfont.so.0 libselinux.so.1
> libconsole.so.0 libuuid.so.1
> libctutils.so.0 libvolume_id.so.0
> libdevmapper.so.1.02.1 modules/
> libdl.so.2 udev/
Apparently the initramfs contains _two_ different libc implementations.
...
[]
> Could we build stripped down versions of those tools (and libs as
> required) build against klibc? I certainly see no need for ncurses in my
> initramfs. Building a klibc based initramfs that then includes libc6
> (and even busybox) as well seems rather stupid. One without klibc would
> be smaller then.
May I ask this question the other way around:
Why maintain two sets of tools and libraries while one set is more
than enough already? Why we need/want klibc to start with, while
regular glibc and regular tools do the work just fine?
I use glibc-based initramfs (with busybox) since first days when
initramfs were introduced in kernel. I booted even the less capable
machines (i486 with 16Mb memory) with it without any issues. I don't
see any reason to maintain another set of tools just for initramfs.
In previous life there was an argument about whole thing fitting a
single floppy drive. But nowadays a) floppies are gone, and b)
kernel itself does not fit in a floppy anymore.
Also, I heard people saying that klibc-based initramfs is somehow
faster than glibc-based. Maybe this is partially true, because the
bigger the initramfs is, the more time it requires to load by a
relatively dumb and slow boot loader (esp. for slow network-based
boots). But even here, in most cases the difference is small and
does not justify the amount of work needed to support the second
set of tools/libraries.
Just an opinion....
/mjt
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/4B7FB03D...@msgid.tls.msk.ru
Goswin von Brederlow
The reason would be size. I don't see anything else there.
For network based boots, specifically high performance cluster, the size
can make a real difference. When you turn the cluster on it is not just
one system downloading an extra meg but 100+ nodes. That largely
increases the network collisions, errors and dropped packages. Something
that can even make systems fail to boot.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87bpfk6...@frosties.localdomain
Marco d'Itri
> For network based boots, specifically high performance cluster, the size
> can make a real difference. When you turn the cluster on it is not just
> one system downloading an extra meg but 100+ nodes. That largely
> increases the network collisions, errors and dropped packages. Something
> that can even make systems fail to boot.
Goswin, meet mtftp. mtftp, meet Goswin.
Also: if the network fabric of your cluster cannot handle an extra 100
MB of boot downloads then it looks badly broken to me.
Do you have any better argument or at least better data? If nobody does,
then I agree with Michael than klibc should be dropped (this way we
*would* save space...).
--
ciao,
Marco
Michael Banck
> Michael Tokarev <m...@tls.msk.ru> writes:
> > Goswin von Brederlow wrote:
> >> I googled a bit and found this old mail about a klibc only initramfs:
> >>
> >> http://lists.debian.org/debian-kernel/2006/07/msg00400.html
> >>
> >> I would really like to do this and it has been close to 4 years since
> >> that mail. But it doesn't look like there has been much progress or not
> >> in the right direction. Looking at my initramfs I see:
[...]
> >> Could we build stripped down versions of those tools (and libs as
> >> required) build against klibc? I certainly see no need for ncurses in my
> >> initramfs. Building a klibc based initramfs that then includes libc6
> >> (and even busybox) as well seems rather stupid. One without klibc would
> >> be smaller then.
> >
> > May I ask this question the other way around:
> >
> > Why maintain two sets of tools and libraries while one set is more
> > than enough already? Why we need/want klibc to start with, while
> > regular glibc and regular tools do the work just fine?
[...]
> The reason would be size. I don't see anything else there.
How about another idea: take advantage of our switch to eglibc and offer
a stripped-down (no i18n, unicode, funky string handling, whatever)
flavour of glibc which could be used in place for this. Another
use-case might be the udeb for debian-installer (though I guess i18n is
important there).
Maybe this has been pondered already or maybe it is already in place,
CCing -glibc.
Michael
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20100220122...@nighthawk.chemicalconnection.dyndns.org
Philipp Kern
> For network based boots, specifically high performance cluster, the size
> can make a real difference. When you turn the cluster on it is not just
> one system downloading an extra meg but 100+ nodes. That largely
> increases the network collisions, errors and dropped packages. Something
> that can even make systems fail to boot.
Network collisions? That sounds so half duplex and wrong to me. Nowadays
we have switches.
Kind regards,
Philipp Kern
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/slrnhnvq19...@kelgar.0x539.de
Steve Langasek
> For network based boots, specifically high performance cluster, the size
> can make a real difference. When you turn the cluster on it is not just
> one system downloading an extra meg but 100+ nodes. That largely
> increases the network collisions, errors and dropped packages. Something
> that can even make systems fail to boot.
Has anyone tried using library reduction to get a stripped down eglibc for
inclusion in initramfs at runtime, using the same techniques as d-i to get a
library that contains only those symbols needed by the included utilities?
I think that's going to be a more maintainable solution than continually
chasing the curve with klibc, both because mklibs is already maintained by
the installer team and because there are always going to be new features
that users are trying to include in their initramfs that will undermine the
effectiveness of the klibc strategy. With mklibs, if you add more utils to
the initramfs the size degrades gracefully; if you use klibc, you get a
sudden jump the first time you have to include something that needs glibc,
and then klibc itself becomes dead weight.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slan...@ubuntu.com vor...@debian.org
Goswin von Brederlow
> On Feb 20, Goswin von Brederlow <goswi...@web.de> wrote:
>
>> For network based boots, specifically high performance cluster, the size
>> can make a real difference. When you turn the cluster on it is not just
>> one system downloading an extra meg but 100+ nodes. That largely
>> increases the network collisions, errors and dropped packages. Something
>> that can even make systems fail to boot.
> Goswin, meet mtftp. mtftp, meet Goswin.
mtftp meet switch. Switch, hey switch, why did you crash?
> Also: if the network fabric of your cluster cannot handle an extra 100
> MB of boot downloads then it looks badly broken to me.
>
> Do you have any better argument or at least better data? If nobody does,
> then I agree with Michael than klibc should be dropped (this way we
> *would* save space...).
There must have been some arguments for it. Someone went through all the
trouble of chaning initramfs-tools to use klibc. The size was just one
example.
The only problem I see is that the change is stuck half way. It either
needs to go forward to completly klibc or back to just eglibc.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87tytau...@frosties.localdomain
Goswin von Brederlow
> On 2010-02-20, Goswin von Brederlow <goswi...@web.de> wrote:
>> For network based boots, specifically high performance cluster, the size
>> can make a real difference. When you turn the cluster on it is not just
>> one system downloading an extra meg but 100+ nodes. That largely
>> increases the network collisions, errors and dropped packages. Something
>> that can even make systems fail to boot.
>
> Network collisions? That sounds so half duplex and wrong to me. Nowadays
> we have switches.
>
> Kind regards,
> Philipp Kern
When 100 nodes all want to talk to the one bootserver then that one poor
port will be overflown. With switches you won't have collisions like in
the old days when they would combine exponentially but you still get
slowdowns.
MfG
Goswin
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/87pr3yu...@frosties.localdomain
Bernd Zeimetz
> When 100 nodes all want to talk to the one bootserver then that one poor
> port will be overflown. With switches you won't have collisions like in
> the old days when they would combine exponentially but you still get
> slowdowns.
Add more switches. Add more network cards to your boot-server. Add a second
bootserver if necessary. Or just don't boot all machines at the same time (hint:
that might also save your building's power supply...).
--
Bernd Zeimetz Debian GNU/Linux Developer
http://bzed.de http://www.debian.org
GPG Fingerprints: 06C8 C9A2 EAAD E37E 5B2C BE93 067A AD04 C93B FF79
ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Wouter Verhelst
> On Sat, Feb 20, 2010 at 12:02:24PM +0100, Goswin von Brederlow wrote:
> > The reason would be size. I don't see anything else there.
>
> > For network based boots, specifically high performance cluster, the size
> > can make a real difference. When you turn the cluster on it is not just
> > one system downloading an extra meg but 100+ nodes. That largely
> > increases the network collisions, errors and dropped packages. Something
> > that can even make systems fail to boot.
>
> Has anyone tried using library reduction to get a stripped down eglibc for
> inclusion in initramfs at runtime, using the same techniques as d-i to get a
> library that contains only those symbols needed by the included utilities?
Don't know, but I'm not sure that's a very good idea. The library
reduction as implemented by mklibs is a bit of a kludge IMO, which seems
to fail every so often for one of our architectures, because that
architecture's ABI uses some obscure ELF feature or other that the
mklibs authors didn't know about, or mklibs missed a symbol dependency,
or something else.
This is not really a big deal in the case of d-i, since first, when
things fail, they fail for everyone who uses the same image, and second,
if the installer fails, you just can't install using the broken image,
but you can still use the system that's already installed (if any), or
fall back to a different install image. Both are not true for initramfs
generation.
--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20100228110...@celtic.nixsys.be
Steve Langasek
> Don't know, but I'm not sure that's a very good idea. The library
> reduction as implemented by mklibs is a bit of a kludge IMO, which seems
> to fail every so often for one of our architectures, because that
> architecture's ABI uses some obscure ELF feature or other that the
> mklibs authors didn't know about, or mklibs missed a symbol dependency,
> or something else.
> This is not really a big deal in the case of d-i, since first, when
> things fail, they fail for everyone who uses the same image, and second,
> if the installer fails, you just can't install using the broken image,
> but you can still use the system that's already installed (if any), or
> fall back to a different install image. Both are not true for initramfs
> generation.
You could obviously just fall back to using the full .so in the case of
initramfs generation.
Wouter Verhelst
> On Sun, Feb 28, 2010 at 12:05:03PM +0100, Wouter Verhelst wrote:
> > This is not really a big deal in the case of d-i, since first, when
> > things fail, they fail for everyone who uses the same image, and second,
> > if the installer fails, you just can't install using the broken image,
> > but you can still use the system that's already installed (if any), or
> > fall back to a different install image. Both are not true for initramfs
> > generation.
>
> You could obviously just fall back to using the full .so in the case of
> initramfs generation.
You could always do that, but it would rather defeat the purpose.
Also, changing back to a working initramfs may be rather hard, depending
on the bootloader in use.
--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/20100301010...@celtic.nixsys.be
Adam Conrad
> On Sun, Feb 28, 2010 at 10:26:48AM -0800, Steve Langasek wrote:
> >
> > You could obviously just fall back to using the full .so in the case of
> > initramfs generation.
If we can detect that the libc generated is unsuitable, then this
fallback seems safe enough.
> Also, changing back to a working initramfs may be rather hard, depending
> on the bootloader in use.
This argument holds for just about any change (epecially ones that we
suspect might behave different on different architectures, with
different bootloaders, or with different underlying storage techniques),
and the answer is the same in all cases. Test, test, test.
I know it's nowhere near as true as it was a decade ago, but "people
running unstable should know how to fix something as simple as their
system failing to boot due to a broken bootloader/kernel/initrd".
Personally, I think the idea of cutting out the klibc duplication and
stripping eglibc during initramfs generation is a pretty reasonable
solution to this longstanding issue.
The extra upshot of this is that some of the weirder corner cases you
refer to that have bitten d-i in the past (A) have made the implementation
more robust, but more interestingly (B) new and similar issues will be
discovered and fixed more readily if this is being tested by more than
just a handful of installer builders/testers.
... Adam
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Wouter Verhelst
(long time no see!)
On Sun, Mar 07, 2010 at 04:13:56PM +0000, Adam Conrad wrote:
> On Mon, Mar 01, 2010 at 02:00:31AM +0100, Wouter Verhelst wrote:
> > On Sun, Feb 28, 2010 at 10:26:48AM -0800, Steve Langasek wrote:
> > >
> > > You could obviously just fall back to using the full .so in the case of
> > > initramfs generation.
>
> If we can detect that the libc generated is unsuitable, then this
> fallback seems safe enough.
The main problem is that this is indeed not easy to detect (otherwise
this would be a non-issue).
[...]
> The extra upshot of this is that some of the weirder corner cases you
> refer to that have bitten d-i in the past (A) have made the implementation
> more robust, but more interestingly (B) new and similar issues will be
> discovered and fixed more readily if this is being tested by more than
> just a handful of installer builders/testers.
That much, certainly, is true. I guess it is the better argument to
indeed go for this option, though I would still suggest doing so in
experimental first.
--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html
--
To UNSUBSCRIBE, email to debian-dev...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org
Archive: http://lists.debian.org/2010030723...@celtic.nixsys.be