Slicing for QEMU

[Will Senn]

All,

I've been working on getting QEMU working reliably with Minix, from
booting with ROOT.MNX to setting up the network. Booting, no problem.
Networking, seems to require /dev/tun which I don't see anywhere, so I'll
leave that alone for now - not too worried about network stuff at the
moment anyway. It's the post boot, setup phase that is giving problems,
at the moment. In particular, I was able to boot the Minix root floppy
and copy it and the usr floppy,and setup USR.TAZ onto an empty harddisk
image, but when I tried to install the SYS.TAZ file, I ran into
difficulty. Apparently, QEMU is sensitive to trash being present in
floppy images. When I tried the following:

on the linux and/or windows host:
dd if=SYS.TAZ of=SYS.00 bs=1440k count=1 skip=0
dd if=SYS.TAZ of=SYS.01 bs=1440k count=1 skip=1

on the minix guest:
setup /usr
or
dd if=/dev/fd0 of=SYS.00
dd if=/dev/fd0 of=SYS.01
cat SYS.00 SYS.01 > SYS.tar.Z
uncompress SYS.tar.Z
tar xvf SYS.tar.Z

I got a "Tar: header checksum error" when reading the second slice.

Long story shorter, I figured out that the second image is smaller than
1440k and when reading that image from the floppy, QEMU chokes on the
trailing garbage.

I then figured out this solution, with the help of a number of folks on
#mfs and #qemu:
cat SYS.TAZ /dev/zero |dd of=SYS.00 bs=1k count=1440 skip=0
cat SYS.TAZ /dev/zero |dd of=SYS.01 bs=1k count=1440 skip=1440

These commands result in 1440k images that are zero filled after reading
from SYS.TAZ. Obviously, the first command could remain as above, I
changed it in the interest of consistency and simplicity.

QEMU had no problem after this.

I would have loved it if the version below had worked, because the math
is easier, but it no worky - something about pipes and 4k limits. Results
in totally erratic behavior - file is a different size nearly every run.
Feel free to explain why:
NO WORKY: cat SYS.TAZ /dev/zero |dd of=SYS.00 bs=1440k count=1 skip=0
NO WORKY: cat SYS.TAZ /dev/zero |dd of=SYS.01 bs=1440k count=1 skip=1

What I would like to know is this: Is it the trailing garbage that causes
the problem or is it a boundary issue with sectors, or what? Does this
seem like a reasonable approach to solving the problem?

Thanks,

Will

[Martijn van Buul]

It occurred to me that Will Senn wrote in comp.os.minix:

> dd if=/dev/fd0 of=SYS.00
> dd if=/dev/fd0 of=SYS.01
> cat SYS.00 SYS.01 > SYS.tar.Z
> uncompress SYS.tar.Z
> tar xvf SYS.tar.Z

Eh, I hope the tar xvf line was a typo? It should read SYS.tar, as uncompress
strips the .Z.

You could use

cat SYS.00 SYS.01 | zcat | tar xvf -

and save yourself the temporary .Z and .tar files. IIRC, zcat is less sensitive
to errors.

> cat SYS.TAZ /dev/zero |dd of=SYS.00 bs=1k count=1440 skip=0
> cat SYS.TAZ /dev/zero |dd of=SYS.01 bs=1k count=1440 skip=1440

> I would have loved it if the version below had worked, because the math
> is easier, but it no worky - something about pipes and 4k limits. Results
> in totally erratic behavior - file is a different size nearly every run.
> Feel free to explain why:
> NO WORKY: cat SYS.TAZ /dev/zero |dd of=SYS.00 bs=1440k count=1 skip=0
> NO WORKY: cat SYS.TAZ /dev/zero |dd of=SYS.01 bs=1440k count=1 skip=1
>

You're writing these images on a Linux host? In that case, the required
temporary memory space by dd (2880K) isn't the issue. WHat *is* the problem
is that you specify the *input* blocksize too - You require it to be
1440K. The default pipe size is 4K, so that is the maximum blocksize you
can use to read from a pipe is 4096 bytes. A bigger read than that will
result in a partial read. There's no really nice workaround.

However, there might be a nice solution to your original problem; you could
try

dd if=SYS.TAZ of=SYS.01 bs=1440k count=1 skip=1 conv=sync

the 'conv=sync' flag should pad the last partial read with nul-bytes up to
the input buffer size (which you defined to be 1440K). It should work,
but I never tried this.

As to the reason of your problems: It might be a Linux issue. I don't know
Linux that well, and I don't know whether a partial write to the floppy
device (of say, 100 bytes) will still cause a sector to be written. I wouldn't
be overly surprised if it wasn't - in which case the last few bytes of your
.TAZ file will never get written to disk, making compress fail to correctly
uncompress the last tar blocks.

For instance, under *BSD, there's two kind of floppy devices: /dev/fd* and
/dev/rfd*. The first one is a block device, the last one is a character
device. Contrary to what you would expect, all writes to the character
device *need* to be done using a multiple of 256 bytes, otherwise you'll loose
data. The blockdevice doesn't suffer from this. As a result,

cat foo.taz > /dev/rfd0a

will not (always) work,

while

cat foo.taz > /dev/fd0a

should.

--
Martijn van Buul - Pie...@c64.org - http://www.stack.nl/~martijnb/
Geek code: G-- - Visit OuterSpace: mud.stack.nl 3333
The most exciting phrase to hear in science, the one that heralds new
discoveries, is not 'Eureka!' (I found it!) but 'That's funny ...' Isaac Asimov

[Will Senn]

Martijn van Buul <pie...@c64.org> wrote in
news:slrncegmie....@mud.stack.nl:

> It occurred to me that Will Senn wrote in comp.os.minix:
>> dd if=/dev/fd0 of=SYS.00
>> dd if=/dev/fd0 of=SYS.01
>> cat SYS.00 SYS.01 > SYS.tar.Z
>> uncompress SYS.tar.Z
>> tar xvf SYS.tar.Z
>
> Eh, I hope the tar xvf line was a typo? It should read SYS.tar, as
> uncompress strips the .Z.

Oops! You are correct, I mistyped in my post.

>
> You could use
>
> cat SYS.00 SYS.01 | zcat | tar xvf -
>
> and save yourself the temporary .Z and .tar files. IIRC, zcat is less
> sensitive to errors.

Yep, that'll work - not sure what IIRC means or what the ramifications of
using zcat are, but seems like a reasonable approach.

> You're writing these images on a Linux host? In that case, the
> required temporary memory space by dd (2880K) isn't the issue. WHat
> *is* the problem is that you specify the *input* blocksize too - You
> require it to be 1440K. The default pipe size is 4K, so that is the
> maximum blocksize you can use to read from a pipe is 4096 bytes. A
> bigger read than that will result in a partial read. There's no really
> nice workaround.
>
> However, there might be a nice solution to your original problem; you
> could try
>
> dd if=SYS.TAZ of=SYS.01 bs=1440k count=1 skip=1 conv=sync
>
> the 'conv=sync' flag should pad the last partial read with nul-bytes
> up to the input buffer size (which you defined to be 1440K). It should
> work, but I never tried this.

This is what I was hoping to hear. Worked beautifully - perfection.
Thanks.

>
> As to the reason of your problems: It might be a Linux issue. I don't
> know Linux that well, and I don't know whether a partial write to the
> floppy device (of say, 100 bytes) will still cause a sector to be
> written. I wouldn't be overly surprised if it wasn't - in which case
> the last few bytes of your .TAZ file will never get written to disk,
> making compress fail to correctly uncompress the last tar blocks.

Sweet, how's it work in *BSD? Didn't occur to me to try in FreeBSD or
Solaris both of which I have available, although not easily accessible.

>
> For instance, under *BSD, there's two kind of floppy devices: /dev/fd*
> and /dev/rfd*. The first one is a block device, the last one is a
> character device. Contrary to what you would expect, all writes to the
> character device *need* to be done using a multiple of 256 bytes,
> otherwise you'll loose data. The blockdevice doesn't suffer from this.
> As a result,
>
> cat foo.taz > /dev/rfd0a
>
> will not (always) work,
>
> while
>
> cat foo.taz > /dev/fd0a
>
> should.
>

Interesting. Thanks Martijn.

-Will

[Martijn van Buul]

It occurred to me that Will Senn wrote in comp.os.minix:
>

> Yep, that'll work - not sure what IIRC means

If I Recall Correctly

http://www.acronymfinder.com/af-query.asp?String=exact&Acronym=IIRC&Find=Find