Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

A well-behaved bootloader

41 views
Skip to first unread message

James Harris

unread,
May 15, 2013, 1:15:17 PM5/15/13
to
We've discussed bootloaders here in the past and I remember
complaining about some of the popular ones because the boot process of
the whole machine becomes dependent on the partition that contains one
OS they are written for.

I came across a comment earlier today about Extlinux (a variant of
Syslinux) that it was "well-behaved" because it sits in a single
partition.

http://www.syslinux.org/wiki/index.php/EXTLINUX

IIUC this means that the MBR simply needs the partition containing
Extlinux to be the one flagged as bootable. That allows other
partitions' VBRs to be made bootable simply by moving the MBR's boot
flag.

So, what do you think of it? Is it a good scheme or a bad scheme and
would you say it is "well-behaved"?

James

Ivan Shmakov

unread,
May 16, 2013, 4:47:34 AM5/16/13
to
>>>>> James Harris <james.h...@gmail.com> writes:

> We've discussed bootloaders here in the past and I remember
> complaining about some of the popular ones because the boot process
> of the whole machine becomes dependent on the partition that contains
> one OS they are written for.

AIUI, it's not the case for GRUB. At the very least, back in
the GRUB 0.XX days, I've used to copy its files to whatever
partition I saw fit, boot GRUB itself from a floppy, and
finalize its installation from there.

That is, GRUB doesn't require a specific OS; it just needs that
its files are somehow copied to a filesystem it has support for.

> I came across a comment earlier today about Extlinux (a variant of
> Syslinux) that it was "well-behaved" because it sits in a single
> partition.

> http://www.syslinux.org/wiki/index.php/EXTLINUX

> IIUC this means that the MBR simply needs the partition containing
> Extlinux to be the one flagged as bootable. That allows other
> partitions' VBRs to be made bootable simply by moving the MBR's boot
> flag.

There, GRUB may be able to "chainload" the partition's
bootloaders without ever requiring the user to modify the MBR.

[...]

--
FSF associate member #7257

James Harris

unread,
May 16, 2013, 8:16:49 AM5/16/13
to
On May 16, 9:47 am, Ivan Shmakov <oneing...@gmail.com> wrote:
> >>>>> James Harris <james.harri...@gmail.com> writes:
>
>  > We've discussed bootloaders here in the past and I remember
>  > complaining about some of the popular ones because the boot process
>  > of the whole machine becomes dependent on the partition that contains
>  > one OS they are written for.
>
>         AIUI, it's not the case for GRUB.  At the very least, back in
>         the GRUB 0.XX days, I've used to copy its files to whatever
>         partition I saw fit, boot GRUB itself from a floppy, and
>         finalize its installation from there.
>
>         That is, GRUB doesn't require a specific OS; it just needs that
>         its files are somehow copied to a filesystem it has support for.

For sure, there are different versions of grub and some may work
differently but AIUI grub requires its own MBR as well as making
changes in the partition used by an OS. Does that seem about right?

>
>  > I came across a comment earlier today about Extlinux (a variant of
>  > Syslinux) that it was "well-behaved" because it sits in a single
>  > partition.
>
>  >  http://www.syslinux.org/wiki/index.php/EXTLINUX
>
>  > IIUC this means that the MBR simply needs the partition containing
>  > Extlinux to be the one flagged as bootable.  That allows other
>  > partitions' VBRs to be made bootable simply by moving the MBR's boot
>  > flag.
>
>         There, GRUB may be able to "chainload" the partition's
>         bootloaders without ever requiring the user to modify the MBR.

But will grub it work with a standard MBR or does it require a
customised one of its own? That's partly rhetorical. I believe all
versions of grub write their own MBRs.

By contrast, Extlinux sits in the VBR and file system of one
partition. That leaves the MBR to be a simple one that just selects a
VBR to load. I'm not saying Extlinux is perfect but it seems to be an
improvement on grub because of being lighterweight and more modular.

James

Ivan Shmakov

unread,
May 16, 2013, 12:14:21 PM5/16/13
to
>>>>> James Harris <james.h...@gmail.com> writes:
>>>>> On May 16, 9:47 am, Ivan Shmakov <oneing...@gmail.com> wrote:
>>>>> James Harris <james.harri...@gmail.com> writes:

>>> We've discussed bootloaders here in the past and I remember
>>> complaining about some of the popular ones because the boot process
>>> of the whole machine becomes dependent on the partition that
>>> contains one OS they are written for.

>> AIUI, it's not the case for GRUB. At the very least, back in the
>> GRUB 0.XX days, I've used to copy its files to whatever partition I
>> saw fit, boot GRUB itself from a floppy, and finalize its
>> installation from there.

>> That is, GRUB doesn't require a specific OS; it just needs that its
>> files are somehow copied to a filesystem it has support for.

> For sure, there are different versions of grub and some may work
> differently but AIUI grub requires its own MBR

The usual MBR-ish GRUB install will use the MBR (for stage1), as
well as the rest of the 0-th track, for the filesystem support
code (AKA stage 1.5.) The rest of it resides on the filesystem.

> as well as making changes in the partition used by an OS. Does that
> seem about right?

My point is that one can use whatever partition for GRUB -- be
it an "OS" one, a "user" one, the one one's storing his or her
videos or photos. Or an entirely dedicated one, just for GRUB
itself. (And, possibly, the kernel images it boots.)

Naturally, one can install GRUB from one OS, and then
reconfigure or update it from another.

[...]

>>> IIUC this means that the MBR simply needs the partition containing
>>> Extlinux to be the one flagged as bootable. That allows other
>>> partitions' VBRs to be made bootable simply by moving the MBR's
>>> boot flag.

>> There, GRUB may be able to "chainload" the partition's bootloaders
>> without ever requiring the user to modify the MBR.

> But will grub it work with a standard MBR or does it require a
> customised one of its own? That's partly rhetorical. I believe all
> versions of grub write their own MBRs.

> By contrast, Extlinux sits in the VBR and file system of one
> partition. That leaves the MBR to be a simple one that just selects
> a VBR to load. I'm not saying Extlinux is perfect but it seems to be
> an improvement on grub because of being lighterweight and more
> modular.

I don't quite understand how a 512-bytes (at most) "head" of
Extlinux can locate its tail residing on the filesystem.
Does it hardcode the tail's location, for instance?

Such a behavior is known to fail should the user, or the system,
relocate the part in question, or attempt to update it, without
running the installer afterwards. Which, in particular, means
that if one uses Extlinux to boot a GNU/Linux system, and it
breaks, one has to use a GNU/Linux to restore the bootloader.
(Compare to GRUB, which may be restored from any OS, provided
that one has support for the filesystem GRUB resides on.)

Nevertheless, this feature actually /is/ available for GRUB (see
"blocklist"), yet strongly advised against.

But these days, this question turns to be a largely rhetorical
one, indeed. I wonder if there still any reason to play the
"trick or four partitions" MBR game? (or to address one's
partitions with CHS, either.) Personally, I've just switched to
GPT wherever possible.

James Harris

unread,
May 17, 2013, 4:34:22 AM5/17/13
to
On May 16, 5:14 pm, Ivan Shmakov <oneing...@gmail.com> wrote:

...

>  > For sure, there are different versions of grub and some may work
>  > differently but AIUI grub requires its own MBR
>
>         The usual MBR-ish GRUB install will use the MBR (for stage1), as
>         well as the rest of the 0-th track, for the filesystem support
>         code (AKA stage 1.5.)

AIUI that only applied to old Grub. It was never a good idea because
nothing regulates use of the rest of the first track or cylinder after
the MBR. Grub 2, I believe, has taken a different approach.

>  The rest of it resides on the filesystem.
>
>  > as well as making changes in the partition used by an OS.  Does that
>  > seem about right?
>
>         My point is that one can use whatever partition for GRUB -- be
>         it an "OS" one, a "user" one, the one one's storing his or her
>         videos or photos.  Or an entirely dedicated one, just for GRUB
>         itself.  (And, possibly, the kernel images it boots.)

Are you saying there could be a small partition with no OS but only
Grub on it and that it could boot Linux and other systems in other
partitions? That would be a considerable improvement, IMHO.

>         Naturally, one can install GRUB from one OS, and then
>         reconfigure or update it from another.

So if, say, Grub were on an Ext2 filesystem do you mean there are
Windows or DOS/bootfloppy utilities that could be used to reconfigure
it?

...

>  > By contrast, Extlinux sits in the VBR and file system of one
>  > partition.  That leaves the MBR to be a simple one that just selects
>  > a VBR to load.  I'm not saying Extlinux is perfect but it seems to be
>  > an improvement on grub because of being lighterweight and more
>  > modular.
>
>         I don't quite understand how a 512-bytes (at most) "head" of
>         Extlinux can locate its tail residing on the filesystem.
>         Does it hardcode the tail's location, for instance?

I don't know. I thought it did hardcode something but cannot find the
reference.

>         Such a behavior is known to fail should the user, or the system,
>         relocate the part in question, or attempt to update it, without
>         running the installer afterwards.

I agree it's not a good approach but according to the following link
that is what *Grub* does.

http://www.gnu.org/software/grub/manual/html_node/Images.html

To quote:

"boot.img cannot understand any file system structure, so grub-setup
hardcodes the location of the first sector of the core image into
boot.img when installing GRUB."

and, re. the next stage, diskboot.img

"Since file system handling is not yet available, it encodes the
location of the core image using a block list format."

>  Which, in particular, means
>         that if one uses Extlinux to boot a GNU/Linux system, and it
>         breaks, one has to use a GNU/Linux to restore the bootloader.

I don't know what you mean "use a GNU/Linux"!

Are you sure in your claim about Extlinux? You may be correct but I
cannot verify it.

Just thinking about it, though, if you had Extlinux installed in a
partition which got trashed or damaged, because Extlinux does not
touch the MBR or need to have any of itself in the MBR you could still
easily set another partition as bootable and boot that.

Alternatively, in the unlikely event that the MBR code got damaged
(not the partition table; if that got lost the disk would need more
attention), with Extlinux you could just plonk (technical term) a
common-or-garden MBR down and set whatever partition you wanted to be
bootable.

It is this lack of dependency on having components in both the MBR and
one partition that I liked about Extlinux and what caused me to start
this thread. There is no such thing as a standard MBR but there are
some "well-behaved" ones that principally do one straightforward
thing: choose a VBR and hand over to it. (I cannot at the moment think
of anything else an MBR has to do. Is there anything?)

Given that the MBR has a simple goal it should be replaceable without
breaking anything else in the system. IIRC even some versions of DOS
or Windows have an fisk /mbr command that replaces the MBR code. It
appears that such an approach would break Grub but not Extlinux.

I started this thread commenting that I liked the clear separation of
MBR and volume boot. This separation of elements is good, IMHO, in
that it removes the interdependency of an overly complex and invasive
system.

>         (Compare to GRUB, which may be restored from any OS, provided
>         that one has support for the filesystem GRUB resides on.)

If I had a machine with Linux running btrfs (I do!) and Grub on the
Linux partition, do you mean that in order to fix that under Windows I
would need a Windows btrfs driver?

>         Nevertheless, this feature actually /is/ available for GRUB (see
>         "blocklist"), yet strongly advised against.

The link I showed above didn't give any alternative to a Grub block
list. What else is there?

IIRC LILO also used a block list.

I don't know what Extlinux does. I have a recollection of finding that
it hardcodes at least one block somewhere (which would not be good
either) but I cannot locate a reference to verify that.

>         But these days, this question turns to be a largely rhetorical
>         one, indeed.  I wonder if there still any reason to play the
>         "trick or four partitions" MBR game? (or to address one's
>         partitions with CHS, either.)  Personally, I've just switched to
>         GPT wherever possible.

Um, I have some broader questions I'd like to ask about that but I'll
not go off topic here. All I'll say here is that even though GPT
formatted disks will become more common it is likely that MBR disks
will continue to exist in the field for a long time to come.

James

Rod Pemberton

unread,
May 17, 2013, 5:51:26 AM5/17/13
to
"James Harris" <james.h...@gmail.com> wrote in message
news:48f22b25-834e-44d0...@q8g2000vbl.googlegroups.com...

> We've discussed bootloaders here in the past [...]

Yes, we have. Sorry, your posts took a few days to get here.

> [...] I remember
> complaining about some of the popular ones because the boot
> process of the whole machine becomes dependent on the
> partition that contains one OS they are written for.
>

"I feel exasperated that Syslinux, Grub and Lilo apparently
load their second stages from fixed locations - often within a
Linux partition or, if not, from somewhere on the disk that
has no protection."

James Harris, "Per-OS boot benefits" thread,
alt.os.development, 4/22/12
news:c280e519-b2d2-4fb3...@e42g2000yqa.googlegroups.com

(I bet you didn't expect that... ;-)

> [...] (a variant of Syslinux) [...]
> http://www.syslinux.org

Does SYSLINUX have a new website? I recall seeing SYSLINUX and
it's numerous other confusing, variant bootloaders, with
inadequate descriptions a few times over the years. However, I
don't recall that website. It's more informative than anything
I've seen on SYSLINUX so far...

> I came across a comment earlier today about Extlinux (a variant
> of Syslinux) that it was "well-behaved" because it sits in a
> single partition.
>
> http://www.syslinux.org/wiki/index.php/EXTLINUX
>

"Note that EXTLINUX installs in the filesystem partition like a
well-behaved bootloader :) Thus, it needs a master boot record in
the partition table; the mbr.bin shipped with SYSLINUX should
work well."

'mbr.bin' in the quote on the EXTLINUX page links to an MBR page:
http://www.syslinux.org/wiki/index.php/Mbr

Wikipedia has some information:
http://en.wikipedia.org/wiki/SYSLINUX

It mentions "Hardware Detection Tool (HDT)". I don't recall
seeing that project. That might be worth a look.
http://hdt-project.org/

> IIUC this means that the MBR simply needs the partition
> containing Extlinux to be the one flagged as bootable. That
> allows other partitions' VBRs to be made bootable simply
> by moving the MBR's boot flag.
>

Yes, possibly...

Unfortunately, the mbr page above doesn't describe the
functionality and operation of mbr.bin very well.

Have you disassembled mbr.bin to confirm what it does?
It's only 512 bytes.

I.e., at this point, I don't know if partition is hardcoded, or if
it selects the partition marked with the boot flag, or if it uses
the boot flag or not... Personally, I'm suspicious that mbr.bin
only selects the first partition. Why? Because, the mbr page
says: 1) altmbr.bin boots a fixed partition, 2) altmbr.bin is only
one byte shorter than mbr.bin, 3) there are three variations of
altmbr.bin, e.g., for the 2nd, 3rd, 4th partitions. The one byte
difference implies to me that mbr.bin and altmbr.bin are actually
the same code, with mbr.bin set to boot the 1st partition. Does
this follow?

> So, what do you think of it?

Well, I've never used it, or the others they provide. AIR, you
seem to prefer VBR code to MBR code, e.g., to allow multiple OS
support.

> Is it a good scheme or a bad scheme [...]?

Yes, it'd be nice if the bootloader selects the partition marked
as active, doesn't attempt to boot partitions not marked as
active, and doesn't hardcode the partition entry the mbr expects
to boot.

> [...] would you say it is "well-behaved"?

What would be considered "badly-behaved" in this instance?

I.e., I'd need to eliminate everything on the "infinite" list of
bad behaviors to say something is "well-behaved". That doesn't
mean that some designs aren't better. Some are. Multi-boot with
it's so-called "a.out kludge" is good for starting compiled C
code.

E.g., are these bad behaviors? What else?
0) doesn't work with values other than CS=0, IP=0x7C00?
1) fails to check if the VBR "code" is bootable or executable?
2) fails to set A20, enable PM, call E820h?
(i.e., some missing 'feature' needed by VBR...)
3) ignores the boot flag?
4) hardcodes the partition to boot?
5) hardcoded filesystem support?
6) hardcoded tracks and sectors?
...

I'm not sure all of these can be solved in a 512 byte bootloader.
E.g., I'm really not sure how a 512 byte bootloader would satisfy
#1 above in all circumstances. If EXTLINUX required a header or
magic value or checksum in the VBR in order to confirm it could
boot, then it could satisfy #1. However, that would deny the user
their choice of VBR, only EXTLINUX compatible VBRs with the
header/checksum/magic value would then work with it. #1 seems to
be dependent on the skill of the user, i.e., doesn't forget to
install a VBR.

> [...] would you say it is "well-behaved"?

(Or, alternate, sarcastic response: WHAT?!?! You would dare to
criticize the revered H.P. Anvin's coding decisions? After all
the foul-ups he's contributed to NASM, no one would dare do that.
Lol...)


Rod Pemberton




wolfgang kern

unread,
May 17, 2013, 12:52:45 PM5/17/13
to

James Harris polled for commtents on bootloaders...

I think it depends on the style of an OS.
If an OS should be capabile of:
execute/run PE/Coff/and other 'oh so common' stuff
then it must support this filesystems as well.
But if an OS got its own filesystem and doesn't need any windoze
and loonix FS then it can have a bootloader which loads the whole
system to either fixed locations or use selfrelocating for modules
and kernel by a previous installed memory manager. My OS uses both,
the kernel structure is fixed at HMA while appended modules may
reside anywhere in RAM.

About split partitions and extra files (loonix SWAP, windoze pagefile):
I never understood the reason for copying data on disk onto disk when
not enough RAM is available. We always can read from origin, so why
make a copy of it to fill another place on disk ? (total nonsense IMW).

My KESYS can reside anywhere in an partition-tree/chain, but the so
called MBR contain already all info found in BR's on other OS.
So I wont waste a full track (historical anyway: 63 sectors) to have
two apart sectors which point to each other.

ie: I call the MBR 'the partition sector' and mine doesn't start with
a jump over data, so the first 256 bytes contain only boot code while
the rest is filled with Volume info, a few text lines and the four
partition entries (even ignored in KESYS-SAFE mode).

Sure I still support other FS (any FAT, CD, partitional NTFS and EXT2)
for read only yet, but this are external optional modules here.

__
wolfgang


James Harris

unread,
May 17, 2013, 3:36:42 PM5/17/13
to
On May 17, 10:51 am, "Rod Pemberton" <do_not_h...@notemailnotq.cpm>
wrote:

...

> > [...] I remember
> > complaining about some of the popular ones because the boot
> > process of the whole machine becomes dependent on the
> > partition that contains one OS they are written for.
>
> "I feel exasperated that Syslinux, Grub and Lilo apparently
> load their second stages from fixed locations - often within a
> Linux  partition or, if not, from somewhere on the disk that
> has no protection."
>
> James Harris, "Per-OS boot benefits" thread,
>  alt.os.development, 4/22/12news:c280e519-b2d2-4fb3...@e42g2000yqa.googlegroups.com
>
> (I bet you didn't expect that...  ;-)

You are right, I didn't! That person talks my language. ;-)

>
> > [...] (a variant of Syslinux) [...]
> >  http://www.syslinux.org
>
> Does SYSLINUX have a new website?  I recall seeing SYSLINUX and
> it's numerous other confusing, variant bootloaders, with
> inadequate descriptions a few times over the years.  However, I
> don't recall that website.  It's more informative than anything
> I've seen on SYSLINUX so far...

I don't think it's new as I found a link to it from a year ago but the
content may have been changed significantly.

> > I came across a comment earlier today about Extlinux (a variant
> > of Syslinux) that it was "well-behaved" because it sits in a
> > single partition.
>
> >  http://www.syslinux.org/wiki/index.php/EXTLINUX
>
> "Note that EXTLINUX installs in the filesystem partition like a
> well-behaved bootloader :) Thus, it needs a master boot record in
> the partition table; the mbr.bin shipped with SYSLINUX should
> work well."
>
> 'mbr.bin' in the quote on the EXTLINUX page links to an MBR page: http://www.syslinux.org/wiki/index.php/Mbr
>
> Wikipedia has some information: http://en.wikipedia.org/wiki/SYSLINUX
>
> It mentions "Hardware Detection Tool (HDT)".  I don't recall
> seeing that project.  That might be worth a look. http://hdt-project.org/

Interesting. It seems the Syslinux family (which is/are really badly
named!) has a com32 plugin interface which others have hooked in to. I
found there is a presentation about Syslinux at

http://www.youtube.com/watch?v=wZB8KxXdiKg

> > IIUC this means that the MBR simply needs the partition
> > containing Extlinux to be the one flagged as bootable. That
> > allows other partitions' VBRs to be made bootable simply
> > by moving the MBR's boot flag.
>
> Yes, possibly...
>
> Unfortunately, the mbr page above doesn't describe the
> functionality and operation of mbr.bin very well.

I don't think it matters much. AIUI that mbr.bin is just something
generic that can be used to replace an MBR which has been trashed or
overwritten.

> Have you disassembled mbr.bin to confirm what it does?
> It's only 512 bytes.

No. I don't think this MBR is too important. AIUI one could use a
Microsoft MBR just as easily.

> I.e., at this point, I don't know if partition is hardcoded, or if
> it selects the partition marked with the boot flag, or if it uses
> the boot flag or not...  Personally, I'm suspicious that mbr.bin
> only selects the first partition.  Why?  Because, the mbr page
> says: 1) altmbr.bin boots a fixed partition, 2) altmbr.bin is only
> one byte shorter than mbr.bin, 3) there are three variations of
> altmbr.bin, e.g., for the 2nd, 3rd, 4th partitions.  The one byte
> difference implies to me that mbr.bin and altmbr.bin are actually
> the same code, with mbr.bin set to boot the 1st partition.  Does
> this follow?

I don't know but I doubt it. There is plenty of space in an MBR to
choose a single bootable partition by boot flag.

HPA seems, though, to have made an effort to cope with those BIOSes
that fail to set DL correctly so there are variants of his MBR.

> > So, what do you think of it?
>
> Well, I've never used it, or the others they provide.  AIR, you
> seem to prefer VBR code to MBR code, e.g., to allow multiple OS
> support.

When we looked at this before (about a year ago as it happens) I
wondered if there was a way to boot linux from a VBR. It turns out
that Extlinux does it. I have replaced Grub with Extlinux on a test
machine and it is fine.

For anyone else who is interested I followed the notes at

http://shallowsky.com/linux/extlinux.html
http://www.syslinux.org/wiki/index.php/EXTLINUX

Because Grub had set up its own MBR I had to overwrite it. I used the
mbr.bin file you mentioned earlier. The surprising thing is that it
can be simply copied to the disk device with

cat mbr.bin > /dev/XXX

Whenever I have seen dd used for suchlike it has included conv=notrunc
so I assumed that a normal file copy would do something more than just
copy bytes. But the command worked and overwrote just the bytes prior
to the partition table keeping the partition table etc intact.

For some reason apt-get didn't seem to recognise anything called Grub
that it could remove so after replacing it and rebooting on Extlinux I
deleted Grub folders and files manually. I think I kept only those
files with grub in the name were part of the package cache. In
particular /boot/grub was removed and, yes, the system still reboots!

Booting with Extlinux doesn't look much different to Grub. In fact I
wasn't sure it had been replaced at first. But the main thing is it
boots Linux and does so from the Linux partition, not from the MBR.
Cool!

Note to self: must remember to try a removable media boot in case the
hard disk Extlinux system gets trashed.

> > Is it a good scheme or a bad scheme [...]?
>
> Yes, it'd be nice if the bootloader selects the partition marked
> as active, doesn't attempt to boot partitions not marked as
> active, and doesn't hardcode the partition entry the mbr expects
> to boot.

I think the Microsoft MBRs do that.

http://thestarman.pcministry.com/asm/mbr/STDMBR.htm

So do lots of others.

> > [...] would you say it is "well-behaved"?
>
> What would be considered "badly-behaved" in this instance?

Terms are often debatable. I would settle for saying it is better
behaved than Grub's because it doesn't interfere with other boot
mechanisms. Different elements can and should be able to work well
together without treading on each other's toes.

This was the old complaint about Microsoft, that they would do their
own thing and never mind working with anyone else. Grub and possibly
Lilo seem similar. Extlinux, while I don't like that it still seems to
load fixed blocks to get its next stage running, does at least play
nicely with other systems.

> I.e., I'd need to eliminate everything on the "infinite" list of
> bad behaviors to say something is "well-behaved".  That doesn't
> mean that some designs aren't better.  Some are.  Multi-boot with
> it's so-called "a.out kludge" is good for starting compiled C
> code.
>
> E.g., are these bad behaviors?  What else?
> 0) doesn't work with values other than CS=0, IP=0x7C00?
> 1) fails to check if the VBR "code" is bootable or executable?
> 2) fails to set A20, enable PM, call E820h?
>      (i.e., some missing 'feature' needed by VBR...)
> 3) ignores the boot flag?
> 4) hardcodes the partition to boot?
> 5) hardcoded filesystem support?
> 6) hardcoded tracks and sectors?
> ...
>
> I'm not sure all of these can be solved in a 512 byte bootloader.
> E.g., I'm really not sure how a 512 byte bootloader would satisfy
> #1 above in all circumstances.  If EXTLINUX required a header or
> magic value or checksum in the VBR in order to confirm it could
> boot, then it could satisfy #1.  However, that would deny the user
> their choice of VBR, only EXTLINUX compatible VBRs with the
> header/checksum/magic value would then work with it.  #1 seems to
> be dependent on the skill of the user, i.e., doesn't forget to
> install a VBR.

I would go in a slightly different direction these days. Now that
secure boot is perceived as an issue it seems better for the MBR to be
able to check that a designated boot partition (which we discussed a
year ago) begins with approved secure code. That way, the chain of
security can be maintained. The security of the boot, if desired, can
be carried into the boot partition.

James

s_dub...@yahoo.com

unread,
May 17, 2013, 3:49:56 PM5/17/13
to
On May 17, 3:34 am, James Harris <james.harri...@gmail.com> wrote:
> On May 16, 5:14 pm, Ivan Shmakov <oneing...@gmail.com> wrote:
>
> ...
>
> I don't know what Extlinux does. I have a recollection of finding that
> it hardcodes at least one block somewhere (which would not be good
> either) but I cannot locate a reference to verify that.
>
> >         But these days, this question turns to be a largely rhetorical
> >         one, indeed.  I wonder if there still any reason to play the
> >         "trick or four partitions" MBR game? (or to address one's
> >         partitions with CHS, either.)  Personally, I've just switched to
> >         GPT wherever possible.
>
> Um, I have some broader questions I'd like to ask about that but I'll
> not go off topic here. All I'll say here is that even though GPT
> formatted disks will become more common it is likely that MBR disks
> will continue to exist in the field for a long time to come.
>
> James

You probably should have started with the broad questions first.

Ivan is coming from familiarity with GPT, and you are not clear about
whether you intend to scale up to using GPT or whether you are just
talking about legacy MBR / VBR.

GPT is just a method of structuring storage, so you can structure any
old HD this way. It sounds like GPT didn't anticipate 4k sectors
(LB's) anyhow. You might want to read up on..

http://en.wikipedia.org/wiki/GUID_Partition_Table

.. which covers some of the issues.

The VBR you're talking about does have a version for GPT's thru the
link 'mbr.bin'. It is also linux filesystem (which-ever) dependent,
its files reside in a directory on the filesystem. The 'well-behaved'
is an off-the-cuff comment, the way I took it. --I don't see where
that mbr.bin is multi-boot, is it? (it looks like it is restricted to
versions of linux for boot selections).

From what I gather from the wiki, GPT doesn't codify a master boot
method.

There could be present, an EFI system partition, which holds the OSen
boot codes ..

http://en.wikipedia.org/wiki/EFI_System_partition

but Apple uses 'Boot Camp':

http://en.wikipedia.org/wiki/Boot_Camp_(software)

The other options are undefined, including a hybrid system of using
LBA 0 as a legacy MBR. That method is outside the scope of GPT, they
say.

From what I gather about GPT, with its many partitions possible, is
that many of those partitions could very well be data only, and not
intended to boot at all. -So one could hold a Multi-boot program's
data, and additional code. -Such as has been said of Grub.

If you are talking about legacy MBR, then you know it's function is to
load the VBR of the one of four partitions marked with the bootable
flag and that is about it. Some check to see if more than one
partition is marked bootable. (Some shoehorn multi-boot menu code in
the rest of track 0). By definition the VBR is tailored to its OS's
filesystem, and so should be 'well-behaved'. 'Well-behaved' has
always meant, in the context of an OS and its utilities, that it
respects that there may be other partitions and OS's on the HD and not
mess with those.

(For me, I'd rather dodge the whole issue by using a dedicated
development machine with a hard drive that only boots the test OS. By
the time I get to the point to have to address the issue, GPT may have
been superceded ;)) )

Steve




James Harris

unread,
May 17, 2013, 4:50:14 PM5/17/13
to
On May 17, 8:49 pm, "s_dubrov...@yahoo.com" <s_dubrov...@yahoo.com>
wrote:

...

> You probably should have started with the broad questions first.

Not really. They only arose when Ivan mentioned GPT. GPT is another
topic.

> Ivan is coming from familiarity with GPT, and you are not clear about
> whether you intend to scale up to using GPT or whether you are just
> talking about legacy MBR / VBR.

"Scale up" is a viewpoint. "Migrate" might be better.

In this thread I was focussing on MBR disks.

...

> The VBR you're talking about does have a version for GPT's thru the
> link 'mbr.bin'.  It is also linux filesystem (which-ever) dependent,
> its files reside in a directory on the filesystem.  The 'well-behaved'
> is an off-the-cuff comment, the way I took it. --I don't see where
> that mbr.bin is multi-boot, is it? (it looks like it is restricted to
> versions of linux for boot selections).

Not sure I follow but you seem to have mentioned two different issues
things in that paragraph.

Extlinux sits wholly within one partition. The partition starts with
the Extlinux VBR and includes further Extlinux code and a Linux
system. Extlinux can boot that version of Linux (and do other things).

It seems that Extlinux does not install anything of itself anywhere
else on disk except in that partition. To me this is a good thing.

However, in case something (like another bootloader) may have not been
so well behaved and may have placed its own code in the MBR a way is
needed to fix that MBR code and get it back to something which just
finds a VBR to boot. Microsoft provides fdisk /mbr and that should be
fine, or one could use mbr.bin.

...

> If you are talking about legacy MBR, then you know it's function is to
> load the VBR of the one of four partitions marked with the bootable
> flag and that is about it.

I believe it *should* do that. Grub and, I think, Lilo do other things
which is why they might need to be overwritten.

...

> ...  'Well-behaved' has
> always meant, in the context of an OS and its utilities, that it
> respects that there may be other partitions and OS's on the HD and not
> mess with those.

Good. I take well-behaved in the same way. In the case of a boot
loader it should work well with other systems.

> (For me, I'd rather dodge the whole issue by using a dedicated
> development machine with a hard drive that only boots the test OS.  By
> the time I get to the point to have to address the issue, GPT may have
> been superceded ;)) )

LOL! Me too.

James

Rod Pemberton

unread,
May 18, 2013, 4:00:48 AM5/18/13
to
"James Harris" <james.h...@gmail.com> wrote in message
news:be5f37fd-459e-4c2f...@j7g2000vbj.googlegroups.com...
> On May 17, 10:51 am, "Rod Pemberton"
<do_not_h...@notemailnotq.cpm>
> wrote:
...

> > (I bet you didn't expect that... ;-)
>
> You are right, I didn't! That person talks my language. ;-)

To quell the creep factor of that a bit, I was searching for an
old post of mine where I thought I mentioned syslinux. Your post
came up too. No my memory is not that good, and I don't have a
big Usenet archive in a database. ;-)

> Because Grub had set up its own MBR I had to overwrite it. I
> used the mbr.bin file you mentioned earlier. The surprising
> thing is that it can be simply copied to the disk device with
>
> cat mbr.bin > /dev/XXX

Apparently, you didn't or hadn't read their Mbr page when you did
that. It says, believe it or not, in direct contradiction to the
line quoted above on their Extlinux page, to NOT use 'cat' to
install an mbr claiming that it's dangerous to use 'cat' for that:

http://www.syslinux.org/wiki/index.php/Mbr

> Whenever I have seen dd used for suchlike it has included
> conv=notrunc so I assumed that a normal file copy would
> do something more than just copy bytes. But the command
> worked and overwrote just the bytes prior to the partition
> table keeping the partition table etc intact.

Honestly, I don't know if 'cat' or 'cp' is safe or not in that
regard... Personally, that's a bit disconcerting to see 'cat'
used for that. I'm confortable with using 'dd' for that. I'd
have to brush up on how 'cat' and 'cp' handle binary data. I'd be
concerned that 'cat' might not handle binary, and 'cp' might or
might not depending on the source or destination. I'd also be
concerned about using 'cat' or 'cp' on a disk that wasn't blank.
I.e., will either overwrite in-use data on a non-blank disk ... ?
I know I can control where 'dd' writes. Generally, I think of
'cat' for displaying text files, 'cp' for copying files, and 'dd'
for writing to devices (or reading). Of course, I don't use Linux
or Unix much, although I've used them both enough over the
years to have a modicum of familiarity.

> Note to self: must remember to try a removable media boot in
> case the hard disk Extlinux system gets trashed.

Yes, I installed Linux on an external USB hard drive a while back.
(Deleted now.) I would use the BBS pop-up menu to boot the
external USB. It would boot from the USB. Linux would get about
half-way through it's mountains of bootup output and fail to
finish booting. I think the kernel was resetting USB during the
bootup, or maybe attempting to reprogram it. (Not good, if so.)
However, loadlin or linld on the Linux kernel from DOS would boot
and mount the USB. So, you might want a floppy or bootable CD
with DOS and loadlin or linld setup to boot your OS. That's
probably much easier than creating a bootable Linux live-cd with
all the recovery utilities you need which also doesn't lock the
boot cdrom drive since it's mounted. Ever try a recovery with
only one cdrom drive, two Linux cd's, one image a bootable live-cd
and the other the recovery utilities? Good luck.

> I would settle for saying [Extlinux] is better behaved than
> Grub's because it doesn't interfere with other boot
> mechanisms.

So, grub interferes? What does it interfere with?

> Different elements can and should be able to work well
> together without treading on each other's toes.
...

> Extlinux, while I don't like that it still seems to
> load fixed blocks to get its next stage running,
> does at least play nicely with other systems.

How would you avoid the fixed blocks issue without installing a
filesystem and providing filesystem support? ...


Rod Pemberton




Rod Pemberton

unread,
May 18, 2013, 4:14:52 AM5/18/13
to
"wolfgang kern" <now...@never.at> wrote in message
news:kn5ncj$cil$1...@newsreader2.utanet.at...

> James Harris polled for commtents on bootloaders...
>
> I think it depends on the style of an OS.
> If an OS should be capabile of:
> execute/run PE/Coff/and other 'oh so common' stuff
> then it must support this filesystems as well.
> But if an OS got its own filesystem and doesn't need any windoze
> and loonix FS then it can have a bootloader which loads the
> whole system to either fixed locations or use selfrelocating for
> modules and kernel by a previous installed memory manager.
> My OS uses both, the kernel structure is fixed at HMA while
> appended modules may reside anywhere in RAM.
>

If a bootloader does not use fixed disk locations, it seems the
bootloader must have filesystem support built-in. You either load
the VBR or 2nd stage bootloader from fixed disk locations, or you
use the filesystem directly to locate a file to load. I get the
impression that James really doesn't like either choice. Both are
too dependent on one thing or another. One uses hardcoded tracks
and sectors. The other needs support for a specific filesystem.
Neither is independent enough for him. I.e., he wants something
entirely independent of a specific filesystem, specific OS, or
hardcoded sectors. I can't see how that's possible.

> About split partitions and extra files (loonix SWAP, windoze
> pagefile): I never understood the reason for copying data on
> disk onto disk when not enough RAM is available. We always
> can read from origin, so why make a copy of it to fill another
> place on disk ? (total nonsense IMW).
>

For Linux, the swap partition is frequently the first partition on
the hard disk. (Isn't it ...? Well, I think it is, generally.)
It's far faster to retrieve data from a small partition at the
start of the hard disk than reading from within a large partition
far away from the start of the disk. Wikipedia calls this
computer science concept "locality of reference". As for Windows,
the pagefile isn't located anywhere special that would offer an
advantage. There are only two advantages I recognize for it. The
first would be if the layout of the data in memory has a different
organization. This would make saving/restoring memory to another
file is better than reloading parts of different files and merging
together. The second could be if the pagefile was located on some
other device than the main storage drive. This could be a fast
ram disk, or a dedicated hard drive. I guess I could read up on
"swap files" myself...

http://en.wikipedia.org/wiki/Locality_of_reference
http://en.wikipedia.org/wiki/Swap_file


Rod Pemberton



Ivan Shmakov

unread,
May 18, 2013, 4:48:37 AM5/18/13
to
>>>>> wolfgang kern <now...@never.at> writes:

[Cross-posting to news:comp.os.linux.misc, news:alt.os.linux,
but leaving the Followup-To: set to news:alt.os.development.]

[...]

> About split partitions and extra files (loonix SWAP, windoze
> pagefile): I never understood the reason for copying data on disk
> onto disk when not enough RAM is available. We always can read from
> origin, so why make a copy of it to fill another place on disk ?
> (total nonsense IMW).

One cannot read the process' working data off disk, because it
isn't there. (Unless it was swapped, that is.)

OTOH, the executables, the shared libraries, and the mmap-ed
files are, to the best of my knowledge, /not/ swapped. As long
as a decent (as in: Linux) virtual memory system is considered.

[...]

The Natural Philosopher

unread,
May 18, 2013, 6:13:36 AM5/18/13
to
I think they can be and are...

And executable has to have a lot of links fixed on loading. a page but
of memory is just that - a ramimage. Its a faster thing to restore than
to reload a program, including its data code pointer and stack.

> [...]
>


--
Ineptocracy

(in-ep-toc’-ra-cy) – a system of government where the least capable to lead are elected by the least capable of producing, and where the members of society least likely to sustain themselves or succeed, are rewarded with goods and services paid for by the confiscated wealth of a diminishing number of producers.

James Harris

unread,
May 18, 2013, 7:36:11 AM5/18/13
to
On May 18, 9:00 am, "Rod Pemberton" <do_not_h...@notemailnotq.cpm>
wrote:

...

> > Because Grub had set up its own MBR I had to overwrite it. I
> > used the mbr.bin file you mentioned earlier. The surprising
> > thing is that it can be simply copied to the disk device with
>
> >   cat mbr.bin > /dev/XXX
>
> Apparently, you didn't or hadn't read their Mbr page when you did
> that.  It says, believe it or not, in direct contradiction to the
> line quoted above on their Extlinux page, to NOT use 'cat' to
> install an mbr claiming that it's dangerous to use 'cat' for that:
>
> http://www.syslinux.org/wiki/index.php/Mbr

I can't remember whether I had read that or not but since this was a
newly built system with no data on it and nothing to lose apart from
the time it took to install I felt like taking the chance. Two web
sites said to do it that way and when tried, it worked.

Notably, at the link you quoted from it goes on to say that dd with
conv=notrunc "prevents the output from truncating the file (if the
output is not a block device, like an image file)." So perhaps the cat
method, while it would damage an image file, is fine with a block
device.

> > Whenever I have seen dd used for suchlike it has included
> > conv=notrunc so I assumed that a normal file copy would
> > do something more than just copy bytes. But the command
> > worked and overwrote just the bytes prior to the partition
> > table keeping the partition table etc intact.
>
> Honestly, I don't know if 'cat' or 'cp' is safe or not in that
> regard...  Personally, that's a bit disconcerting to see 'cat'
> used for that.  I'm confortable with using 'dd' for that.  I'd
> have to brush up on how 'cat' and 'cp' handle binary data.  I'd be
> concerned that 'cat' might not handle binary, and 'cp' might or
> might not depending on the source or destination.

I doubt there's an issue with that. Unix commands, like much of Unix,
usually deal simply with bytes so whether they are text or binary
makes no difference. MS-DOS's copy command had a /b option for binary
files because MS-DOS is not well-behaved :^) Maybe that's what you had
in mind?

Of course, in the cat case it is not cat which is writing to the
output device but the shell's piped redirection but even then I
wouldn't expect Unix to do anything other than process bytes.

>  I'd also be
> concerned about using 'cat' or 'cp' on a disk that wasn't blank.
> I.e., will either overwrite in-use data on a non-blank disk ... ?

Yes, unless I was more sure than I am now I would use dd on a system
that had data I wanted to keep.

...

> > I would settle for saying [Extlinux] is better behaved than
> > Grub's because it doesn't interfere with other boot
> > mechanisms.
>
> So, grub interferes?  What does it interfere with?

Grub takes over almost the whole boot process (as far as it goes). It
doesn't work well with other packages. It puts itself in both the MBR
and a partition - usually (and perhaps always?) a Linux partition.
Extlinux, on the other hand (and I am not a fan of it but I like that
it keeps its mitts out of things that Grub gives an OS developer
problems with) resides wholly within a partition and can thus be
switched on or not as needed without preventing the rest of the boot
process working as designed.

Even Microsoft's boot process, from what I can remember, sits within
its own partition. That's quite surprising. Microsoft, who tend to do
their own thing, work well with other boot components. At least they
used to. Not sure about now.

...

> > Extlinux, while I don't like that it still seems to
> > load fixed blocks to get its next stage running,
> > does at least play nicely with other systems.
>
> How would you avoid the fixed blocks issue without installing a
> filesystem and providing filesystem support? ...

I don't know. It would depend on the filesystem. Some (like the fats
and the exts) allow for a boot area to follow the first sector. It is
better than a boot sector but it may not be large enough for basic
filesystem support. (Fat's is extensible. Cannot remember for the exts
but it might be just 1k.)

It might be possible to provide enough read-only filesystem support in
a limited space. After all, I did it in about 450 bytes for the fairly
awkward fat12 (though the boot file had to sit in the root directory
and I could only use one of the two file allocation tables; there
wasn't enough room for code to fall back to the other one if the first
was damaged). So it might be possible in 1k for most filesystems. But
I would have to try to find out.

Another option is to have an all-knowing boot partition that
understands all the file systems. I don't really like that idea as it
is less modular but it might be worth throwing in to the discussion.
Such a partition could do anything and everything necessary to load
and start kernels of different ilks. I suppose such a boot partition
could have loadable modules to cope with different filesystems and
different kernels. It would certainly make overall boot management
easier but it would also make the system mode dependent on one
partition. Umm, not sure.

James

James Harris

unread,
May 18, 2013, 7:49:03 AM5/18/13
to
On May 18, 9:14 am, "Rod Pemberton" <do_not_h...@notemailnotq.cpm>
wrote:

...

> If a bootloader does not use fixed disk locations, it seems the
> bootloader must have filesystem support built-in.  You either load
> the VBR or 2nd stage bootloader from fixed disk locations, or you
> use the filesystem directly to locate a file to load.  I get the
> impression that James really doesn't like either choice.  Both are
> too dependent on one thing or another.  One uses hardcoded tracks
> and sectors.  The other needs support for a specific filesystem.
> Neither is independent enough for him.  I.e., he wants something
> entirely independent of a specific filesystem, specific OS, or
> hardcoded sectors.  I can't see how that's possible.

Not quite. Like Ivan, I don't like the boot loader requiring something
in the filesystem to be at a fixed location. Something might move it
and then the boot would fail. (Strictly speaking the boot might
continue to work until the original blocks were overwritten. That
could happen any time such as when a disk is defragmented or when a
perfectly innocent file copy was done. And, of course, reboots are not
always frequent. So the boot failure could happen a long time after
the file was moved - i.e. the failure could happen a long time after
the cause of the failure. That does not make a good recipe for happy
users or happy support staff!)

I don't have a problem with a *VBR* understanding enough of a
filesystem to load a file from it. The VBR and filesystem are
unavoidably linked as they are part of the same partition so it seems
reasonable that they should work together.

I don't like the idea of a disk's MBR loading something from inside a
partition, though. Is that so strange an idea? ISTM the MBR should
load and start a VBR and that's all.

Is there a better way?

James

James Harris

unread,
May 18, 2013, 7:54:13 AM5/18/13
to
On May 18, 12:36 pm, James Harris <james.harri...@gmail.com> wrote:

...

> easier but it would also make the system mode dependent on one

Oops, "mode" should have been "more"

James

s_dub...@yahoo.com

unread,
May 19, 2013, 12:15:06 AM5/19/13
to
[legacy MBR]

Well, if you don't mind using up a partition table record, one option
is for the MBR to use it's own partition, albeit a small one, in which
it has the code to present a menu to the user to choose an OS to boot.
(now, one of three possible). MBR is taken to be at C,H,S = 0,0,1 but
there is variance on whether the first partition begins at 0,0,2 or
begins at the cylinder boundry 1,0,1. With using a MBR partition
table entry beginning at 0,0,2 until the end of the cylinder, that
area is protected from other OS installation programs. The menu MBR
avoids the need to have a utility written in each OS to remark for the
next OS's boot flag. If your hobby OS is written to a fresh HD, and
is the first on it, then it can enforce installation to the cylinder
boundry and not use up a partition table record.

BTW, have you looked here lately..

http://en.wikipedia.org/wiki/Master_boot_record

There's a description of a number of MBR's.

Steve

James Harris

unread,
May 20, 2013, 8:39:36 AM5/20/13
to
On May 19, 5:15 am, "s_dubrov...@yahoo.com" <s_dubrov...@yahoo.com>
wrote:

...

> [legacy MBR]
>
> Well, if you don't mind using up a partition table record, one option
> is for the MBR to use it's own partition,

If I understand, then: agreed. That was where I got to in the
following thread:

https://groups.google.com/group/alt.os.development/browse_frm/thread/1d1f6c8a81604777

I.e. a small partition could/should be set aside for traditional (i.e.
non-EFI) boot code. The cost was almost zero as the space needed was
tiny compared with a typical drive's size and the thus-consumed
partition could be replaced by an added EBR.

> albeit a small one, in which
> it has the code to present a menu to the user to choose an OS to boot.
> (now, one of three possible).  MBR is taken to be at C,H,S = 0,0,1 but
> there is variance on whether the first partition begins at 0,0,2 or
> begins at the cylinder boundry 1,0,1.

Except for legacy compatibility (with such as MS-DOS and maybe some
early versions of Windows) I believe it is now recommended no longer
to begin partitions on cylinder boundaries. Cylinder boundaries were
relevant when we accessed disks by physical CHS values but all
accesses have long been translated and we can only use nominal CHS
values. Early translation was mainly to increase capacity but now that
disks are recorded in zones there is not even a consistent set of CHS
values to apply across the medium. Specifically, different tracks have
different numbers of sectors.

Now there is a different issue. Use of the old 512-byte sector size
which has been with us so long is diminishing. Modern drives are
formatted in 4k physical sectors. If those are partitioned at cylinder
boundaries that can mean that each partition begins part-way through a
sector. That could be a bad idea for performance. So drives should
often now be partitioned at boundaries which are multiples of at least
4k from the start of the disk.

Of course, now that the long-standing 512-byte rule has been broken it
is possible that we will see drives with other physical sector sizes
in future so 4k may not be the final word. Perhaps at least partly for
that reason some partitioning tools set partitions further apart than
4k requires. IIRC some set up partitions on 1M boundaries or something
similar. Of course, 1M aligned partitions are also 4k aligned.

> With using a MBR partition
> table entry beginning at 0,0,2 until the end of the cylinder, that
> area is protected from other OS installation programs.

Yes, as long as the space is reserved in the partition table it should
be protected. I don't think anything protects that region otherwise.

> The menu MBR
> avoids the need to have a utility written in each OS to remark for the
> next OS's boot flag.  If your hobby OS is written to a fresh HD, and
> is the first on it, then it can enforce installation to the cylinder
> boundry and not use up a partition table record.

ISTM the use of a partition table entry is essential. There have not
been many but there have been some programs which eyed greedily (!)
the otherwise unallocated space between CHS 0,0,2 and the end of the
cylinder and thought: I can use that. Nothing regulated their treading
on each other.

> BTW, have you looked here lately..
>
> http://en.wikipedia.org/wiki/Master_boot_record
>
> There's a description of a number of MBR's.

It may have been a while since I looked there. If you mean the MBR
partitioning schemes I do remember seeing some which had more than
four entries. The problem with all of those is they are not recognised
by the more mainstream tools. Hence, the extra partition entries could
be silently corrupted. Unless an OS is to run in a customised
environment ISTM better to use the limited but standard four-entry
partitioning scheme. It stands less chance of being corrupted, works
with standard tools and, as mentioned above, can be used in a way that
does not restrict the user's choices.

James
0 new messages