Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Bug#610783: bsdtar: Doesn't extract the install* and isolinux* directories of d-i images

26 views
Skip to first unread message

Samuel Thibault

unread,
Jan 22, 2011, 7:40:01 AM1/22/11
to
Package: bsdtar
Version: 2.8.4-1
Severity: normal

Hello,

(after some lazy patching against bug #610781)

$ bsdtar xf debian-sq-di-rc1-i386-businesscard.iso
$ ls install* isolinux
install:

install.386:

isolinux:
$

It seems libarchive is not able to extract the install and isolinux
directories.

Samuel

-- System Information:
Debian Release: 6.0
APT prefers testing
APT policy: (990, 'testing'), (500, 'unstable'), (500, 'stable'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.37 (SMP w/2 CPU cores)
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/bash

Versions of packages bsdtar depends on:
ii libacl1 2.2.49-4 Access control list shared library
ii libarchive1 2.8.4-1 Single library to read/write tar,
ii libattr1 1:2.4.44-2 Extended attribute shared library
ii libbz2-1.0 1.0.5-6 high-quality block-sorting file co
ii libc6 2.11.2-7 Embedded GNU C Library: Shared lib
ii liblzma2 5.0.0-2 XZ-format compression library
ii libxml2 2.7.8.dfsg-2 GNOME XML library
ii zlib1g 1:1.2.3.4.dfsg-3 compression library - runtime

bsdtar recommends no packages.

Versions of packages bsdtar suggests:
ii bsdcpio 2.8.4-1 cpio(1) from FreeBSD, using libarc

-- no debconf information

--
To UNSUBSCRIBE, email to debian-bugs-...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listm...@lists.debian.org

Samuel Thibault

unread,
Jan 24, 2011, 8:50:02 AM1/24/11
to

Thomas Schmitt

unread,
Jan 24, 2011, 10:40:02 AM1/24/11
to
Hi,

> $ ls install* isolinux
> install:
> install.386:
> isolinux:

I am quite clueless about this.

> $ bsdtar xf debian-sq-di-rc1-i386-businesscard.iso

As learned from Bug#610781, i examine:
http://cdimage.debian.org/cdimage/squeeze_di_rc2/i386/iso-cd/debian-squeeze-di-rc2-i386-businesscard.iso
(is rc1 still availiable ?)

/install is indeed empty. But the other two bear files.
I can see them by mount(8) of my elderly SuSE and by xorriso.

Are there more differences between the tree of the mounted image
and the tree extracted by bsdtar ?

Any idea where libarchive decides in its source code not to extract the
files ?
Is it known whether it uses Rock Ridge, Joliet or dull ISO/ECMA metadata ?
(Distinguishable when extracting long names with mixed case.)


Have a nice day :)

Thomas

Steve McIntyre

unread,
Jan 24, 2011, 10:50:02 AM1/24/11
to
On Mon, Jan 24, 2011 at 04:36:06PM +0100, Thomas Schmitt wrote:
>Hi,
>
>> $ ls install* isolinux
>> install:
>> install.386:
>> isolinux:
>
>I am quite clueless about this.
>
>> $ bsdtar xf debian-sq-di-rc1-i386-businesscard.iso
>
>As learned from Bug#610781, i examine:
> http://cdimage.debian.org/cdimage/squeeze_di_rc2/i386/iso-cd/debian-squeeze-di-rc2-i386-businesscard.iso
>(is rc1 still availiable ?)

http://cdimage.debian.org/cdimage/squeeze_di_rc1/ etc.

--
Steve McIntyre, Cambridge, UK. st...@einval.com
"This dress doesn't reverse." -- Alden Spiess

Samuel Thibault

unread,
Jan 24, 2011, 10:50:02 AM1/24/11
to
Thomas Schmitt, le Mon 24 Jan 2011 16:36:06 +0100, a écrit :
> > $ bsdtar xf debian-sq-di-rc1-i386-businesscard.iso
>
> As learned from Bug#610781, i examine:
> http://cdimage.debian.org/cdimage/squeeze_di_rc2/i386/iso-cd/debian-squeeze-di-rc2-i386-businesscard.iso
> (is rc1 still availiable ?)

rc1 is still available, the name is a bit different however

> /install is indeed empty.

Yes, that one is expected. But install.386 and isolinux are not.

> But the other two bear files. I can see them by mount(8) of my
> elderly SuSE and by xorriso.

> Are there more differences between the tree of the mounted image
> and the tree extracted by bsdtar ?

Mmm, actually there is nothing in _all_ the directories...

Samuel

Andreas Henriksson

unread,
Jan 24, 2011, 11:00:02 AM1/24/11
to
Hello!

On Mon, Jan 24, 2011 at 04:36:06PM +0100, Thomas Schmitt wrote:

> As learned from Bug#610781, i examine:
> http://cdimage.debian.org/cdimage/squeeze_di_rc2/i386/iso-cd/debian-squeeze-di-rc2-i386-businesscard.iso
> (is rc1 still availiable ?)

I guess any of them works to reproduce the problem,
but you can still get it from here:

http://cdimage.debian.org/cdimage/squeeze_di_rc1/i386/iso-cd/debian-sq-di-rc1-i386-businesscard.iso

>
> /install is indeed empty. But the other two bear files.
> I can see them by mount(8) of my elderly SuSE and by xorriso.
>
> Are there more differences between the tree of the mounted image
> and the tree extracted by bsdtar ?
>
> Any idea where libarchive decides in its source code not to extract the
> files ?

(Somewhere in libarchive/archive_read_format_support_iso9660.c is all I can
say for now, but I guess that doesn't help you...)

> Is it known whether it uses Rock Ridge, Joliet or dull ISO/ECMA metadata ?
> (Distinguishable when extracting long names with mixed case.)

unless explicitly instructed to use something, this should be the order
in which it tries to detect and use the extensions:

joliet, rockridge, iso/ecma...


Example, to skip any joliet metadata use:
bsdtar --options !joliet ...

>
>
> Have a nice day :)
>
> Thomas
>
>
>

--
Andreas Henriksson

Thomas Schmitt

unread,
Jan 24, 2011, 11:20:01 AM1/24/11
to
Hi,

Samuel Thibault:


> Mmm, actually there is nothing in _all_ the directories...

This makes it somewhat less puzzling. I was wondering what could
be the decisive difference between /install.386 and others.

Do i get it right that only empty top level directories get
extracted ? I.e. /dists contains no ./squeeze ?


> > Any idea where libarchive decides in its source code not to extract the
> > files ?

Andreas Henriksson:


> (Somewhere in libarchive/archive_read_format_support_iso9660.c is all I can
> say for now, but I guess that doesn't help you...)

I am looking at it and recognize the terms of ECMA-119 et.al.
But i still do not understand how it reads directories.
A candidate the is the "while(step)" loop in read_children().


Andreas Henriksson:


> it tries to detect and use the extensions:
> joliet, rockridge, iso/ecma...

Any success if you outrule Joliet and force Rock Ridge ?


Have a nice day :)

Thomas


--

Samuel Thibault

unread,
Jan 24, 2011, 11:30:02 AM1/24/11
to
Thomas Schmitt, le Mon 24 Jan 2011 17:13:08 +0100, a écrit :
> Andreas Henriksson:
> > it tries to detect and use the extensions:
> > joliet, rockridge, iso/ecma...
>
> Any success if you outrule Joliet and force Rock Ridge ?

I tried

bsdtar -x --options 'iso9660:!joliet,iso9660:rockridge' -f ../debian-squeeze-di-rc2-i386-businesscard.iso

with same result.

Samuel

Samuel Thibault

unread,
Jan 24, 2011, 11:30:02 AM1/24/11
to
Thomas Schmitt, le Mon 24 Jan 2011 17:13:08 +0100, a écrit :
> Do i get it right that only empty top level directories get
> extracted ? I.e. /dists contains no ./squeeze ?

Yes.

Samuel

Thomas Schmitt

unread,
Jan 24, 2011, 11:40:01 AM1/24/11
to
Hi,

i booted the Debian installation of the project test machine
Linux debian 2.6.26-2-amd64 #1 SMP Tue Jan 12 22:12:20 UTC 2010 x86_64
(Yeah, luxury. A dedicated modern test machine. Donated.)

Now i would need instructions for dummies how to create a debuggable
bsdtar resp. one that gives me opportunity to inject a few fprintf.
Internet is reachable from the system. (Not that i could remember
how i did it a year ago.)


Have a nice day :)

Thomas


Samuel Thibault

unread,
Jan 24, 2011, 11:50:02 AM1/24/11
to
Thomas Schmitt, le Mon 24 Jan 2011 17:34:52 +0100, a écrit :
> i booted the Debian installation of the project test machine
> Linux debian 2.6.26-2-amd64 #1 SMP Tue Jan 12 22:12:20 UTC 2010 x86_64
> (Yeah, luxury. A dedicated modern test machine. Donated.)
>
> Now i would need instructions for dummies how to create a debuggable
> bsdtar resp. one that gives me opportunity to inject a few fprintf.
> Internet is reachable from the system. (Not that i could remember
> how i did it a year ago.)

in /etc/apt/sources.list, if you don't have a deb-src line copy/paste
a deb line pointing to an official Debian mirror and replace deb with
deb-src, run apt-get update

apt-get source bsdtar
sudo apt-get build-dep bsdtar
cd bsdtar-foo
DEB_BUILD_OPTIONS="noopt nostrip" dpkg-buildpackage -B

and either run from there, or dpkg -iO ../*.deb

Samuel

Thomas Schmitt

unread,
Jan 24, 2011, 3:00:03 PM1/24/11
to
Hi,

it seems the immediate reason is in
libarchive/archive_read_support_format_iso9660.c:read_entries()
which iterates over
next_entry(iso9660)
until a NULL or a non-directory appears.

With an image created by
mount -o loop /reiser/p/v/debian-squeeze-di-rc2-i386-businesscard.iso /mnt
genisoimage -o debian-businesscard-genisoimage.iso \
-allow-leading-dots -J -R -graft-points /=/mnt
i first see all 158 directories being enumerated and
only then come the other files.
With the image from xorriso i see the symbolic link 'debian' as second
item, which ends the loop. After a few more directories comes the first
regular file.

The iteration sequence with the image from genisoimage is _not_ the
sequence of directory entries in that image.
At least in the root directory the sequences of directory entries
are the same in both images.

So next i will have to find out what determines the iteration sequence.
But first i need some calories.


Have a nice day :)

Thomas


Andreas Henriksson

unread,
Jan 24, 2011, 3:10:04 PM1/24/11
to

----- Original message -----
> Hi,
>
> i booted the Debian installation of the project test machine
>    Linux debian 2.6.26-2-amd64 #1 SMP Tue Jan 12 22:12:20 UTC 2010 x86_64
> (Yeah, luxury. A dedicated modern test machine. Donated.)
>
> Now i would need instructions for dummies how to create a debuggable
> bsdtar resp. one that gives me opportunity to inject a few fprintf.
> Internet is reachable from the system. (Not that i could remember
> how i did it a year ago.)

you might want to start with apt-get update....

Then try:
mkdir /tmp/foobar && cd /tmp/foobar
apt-get source libarchive
cd libarchive-2*
DEB_BUILDGOPTIONS="noopt nostrip debug" dpkg-buildpackage -uc -us

this should spit out .deb packages in /tmp/foobar containing non-optimized debuggage binaries.

You install them with "dpkg -i /tmp/foobar/*.deb"

if you want to inject printfs, just edit the source and rerun from the dpkg-buildpackage step.....

George Danchev

unread,
Jan 24, 2011, 3:40:02 PM1/24/11
to
On Monday 24 January 2011 21:53:50 Thomas Schmitt wrote:

Hi,

(I changed the bug address to 610783@ as this is the bug we are talking about
now)

> it seems the immediate reason is in
> libarchive/archive_read_support_format_iso9660.c:read_entries()
> which iterates over
> next_entry(iso9660)
> until a NULL or a non-directory appears.
>
> With an image created by
> mount -o loop /reiser/p/v/debian-squeeze-di-rc2-i386-businesscard.iso
> /mnt genisoimage -o debian-businesscard-genisoimage.iso \
> -allow-leading-dots -J -R -graft-points /=/mnt
> i first see all 158 directories being enumerated and
> only then come the other files.
> With the image from xorriso i see the symbolic link 'debian' as second
> item, which ends the loop. After a few more directories comes the first
> regular file.
>
> The iteration sequence with the image from genisoimage is _not_ the
> sequence of directory entries in that image.
> At least in the root directory the sequences of directory entries
> are the same in both images.
>
> So next i will have to find out what determines the iteration sequence.
> But first i need some calories.

At the head of that source file I read:

* This module works by first reading the volume descriptors, then
* building a list of directory entries, sorted by starting
* sector. At each step, I look for the earliest dir entry that
* hasn't yet been read, seek forward to that location and read
* that entry. If it's a dir, I slurp in the new dir entries and
* add them to the heap; if it's a regular file, I return the
* corresponding archive_entry and wait for the client to request
* the file body. This strategy allows us to read most compliant
* CDs with a single pass through the data, as required by libarchive.

This entry sequencing smells as an assumption to me. I'm not aware ECMA-119 to
stipulate something like that. Am I missing something?

--
pub 4096R/0E4BD0AB <people.fccf.net/danchev/key pgp.mit.edu>

Thomas Schmitt

unread,
Jan 24, 2011, 5:20:02 PM1/24/11
to
Hi,

Andreas Henriksson:


> DEB_BUILDGOPTIONS="noopt nostrip debug" dpkg-buildpackage -uc -us
> this should spit out .deb packages in /tmp/foobar containing
> non-optimized debuggage binaries.
> You install them with "dpkg -i /tmp/foobar/*.deb"

For now i helped myself with vim and make.
I will follow your advise as soon as i really need gdb. For now i am
studying source and watch my test messages.


----------------------------------------------------------------------
Theory:

Do i get it right that

heap_add_entry(struct heap_queue *heap, struct file_info *file, uint64_t key)
builds a kind of sorted list (= tree) with the files' storage addresses
as key ?
Further, does the recursive building of the tree already begin to traverse it
while it is not complete ?

This would work only if all adding of items happens in the not yet read
part of the tree. This demands a roughly monotonously rising sequence
of storage addresses while traversing the directory tree of the image.

In that case the spoilsport would be our symbolic links which bear storage
address 0 in order not to coincide with a data file's content data.
(A symbolic link is an empty ISO file with the link target stored in
a Rock Ridge field that is part of the directory tree.)


----------------------------------------------------------------------
Verification:

If i force the 0-keys to values which match the needs of my theory, then
i get extracted a tree from the xorriso image.

static void
heap_add_entry(struct heap_queue *heap, struct file_info *file, uint64_t key)
{
uint64_t file_key, parent_key;
int hole, parent;

/* ts 24 Jan 2011:
The storage address of the first file "autorun.inf" is the
lowest safe value. LBA 10849.
*/
static uint64_t high_key = 22218752;

/* ts 24 Jan 2011:
libisofs symbolic links have LBA 0 = key 0.
read_entries() assumes that directories have lower keys than
any non-directories.
*/
if (key == 0)
key = high_key;
else if (key > high_key)
high_key = key;

[... normal code ...]

Regrettably this proof-of-concept is not a general remedy, because i
had to hardcode the storage address of the first data file as start
value for the plausible addresses. (Also it is not thread safe.)

It would not be safe for general ISO images, anyway. The assumptions
are too daring. Program "flyisofs" mixes directories and data.
multi-session images of mkisofs, genisoimage, or xorriso have a
directory chunk, a data chunk, another directory chunk, another data chunk,
and so on.

If my theory is right, then libarchive's ISO code is wrong.

----------------------------------------------------------------------

On the first glimpse astounding is the state of my two unpack directories:

$ du -s bsdtar_genisoimage bsdtar_xorriso
48344 bsdtar_genisoimage
46200 bsdtar_xorriso
$ diff -r bsdtar_genisoimage bsdtar_xorriso
$

Both sum up to 48353975 bytes by ls -lR. That's 47221 kB.

The xorriso image unpacks a hardlink in install.386 and install.386/gtk
-r--r--r-- 2 thomas thomas 2189312 2011-01-21 21:12 vmlinuz
whereas the genisoimage image yields two copied files
-r--r--r-- 1 thomas thomas 2189312 2011-01-21 21:12 vmlinuz

Reason could be our Rock Ridge PX entries with inode numbers.

----------------------------------------------------------------------

I will ponder whether i can deliver symbolic links with LBAs that
match libarchive's expectations.
But honestly, those expectations should rather be dropped.

George Danchev:


> This entry sequencing smells as an assumption to me.

It would be ok. xorriso does a similar sorting when extracting files,
because random access seeking on optical media is darn slow.
Only that it is not so optimistic to begin single-pass traversing
a not yet complete tree and risking to not visit some branches or
leafs.

The comment you found does not mention the daring assumption about
LBAs of directories always being lower than LBAs of other files.


Have a nice day :)

Thomas


Thomas Schmitt

unread,
Jan 25, 2011, 3:40:02 AM1/25/11
to
Hi,

after some more calories and sleep i now see two problems in
libarchive:

- ISO 9660 images can be read in a single pass only if the directory
entry of any file is stored at a lower address than the file's
content.
This makes multi-session image unsuitable for libarchive.
No workaround seems possible.

- The implementation of the ISO image reader assumes that
all directories come before any data file content.
This assumption fails with "flyisofs" images which are elsewise
the ideal counterpart to libarchive. They get produced in a
single pass.
The assumption also fails with libisofs symbolic links and
empty data files, which have dummy content addresses.

So my proposals for libarchive are:

- Detect address pointers which refer to blocks which have already
been read and forgotten. This could happen while building the tree.
I cannot spot such a test and error message yet.
Senseful reactions would then be to abort or to skip the offending
file or directory tree.

- Become able to process directories and other files in mixed
order. This is not easily done but i see no fundamental obstacle
with the single-pass constraint.

- Map eventual keys in the range of [0 , address_of_volume_set_terminator]
to the file's directory's key plus one. The end address of the interval
would be recordable in the function isVDSetTerminator(), which is known
from bug#610781. It is called before any files get read.
A file with address in that interval will never need to read content
from there. Surely promised.


As for libisofs avoiding the 0 keys:

The zeros for symbolic links and device files are written since
libisofs.so.6 was founded by Vreixo Formoso in autumn 2007.

For empty data files the address of the Volume Descriptor Set
Terminator is used since revision 732, 25 Nov 2010. This was in
the course of boot experiments for future Debian images.
I find no hint, though, that this was necessary to make the experiments
succeed. It looks rather like a little overhaul instigated by suboptimal
error behavior with an empty boot image file.
(There are no empty data files in the Debian image.)

It would be more or less harmless to revert the change of rev 732,
but to give non-zero addresses to links and device files will
put in question the hoarded xorriso testing of 3 years.
So in any case, this can be made only a non-default option in xorriso.

I will go for such an option, but it may last a few days.
Debian CD production should sincerely weigh pros and cons of using it.
(I can can give few advise about the risk.)

As said, it would not be necessary if libarchive would change the
implementation of read_entries() so that it can stand a mix of
directories and files, and would map dummy addresses to the directory
key + 1.

Andreas Henriksson

unread,
Jan 25, 2011, 4:50:01 AM1/25/11
to
Hello Thomas!

Thanks for your very thorough investigation on this. There are already
a couple of bugs in the libarchive bug tracker talking about
problems of the streams design of libarchive so I guess this
is just another one to put on the pile unfortunately.

I'm adding Tim Kientzle and the libarchive-discuss list to CC.
Please note this url in case anyone wants to read up on the entire
discussion:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610783

Thomas latest/final analyze included in full below...

--
Andreas Henriksson

Thomas Schmitt

unread,
Jan 25, 2011, 11:10:01 AM1/25/11
to
Hi,

the situation now appears a bit better than first perceived in
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=610783

The demand of libarchive that all directory entries have to come
before any content block eases the task of producing digestible
addresses for symbolic links, device files and empty data files.

It suffices to let all these files point to an arbitrary block
after the directory tree.
In the general situation i would have to make them point to
their neighbors. A much more ill situation, and also much more
demanding towards the current libisofs architecture.

If libarchive ever gives up the demand of directories-first, then
it will have to compute own suitable keys for the affected file
types which have no data content.
Another problem would be hard links, which are quite common
in ISO images. So my change proposal for libarchive appears more
and more cumbersome.

The simpler change in libisofs will probably allow me to make it
suitable for unchanged libarchive by default.
A dedicated block of 2048 zero bytes should avoid any ambiguity
with non-empty data files.
I am currently testing an implementation sketch which looks
quite trustworthy.


Another insight:
The reason why bsdtar with genisoimage did not create two hard links
to vmlinuz is in my Linux. It never shows hardlink siblings in mounted
ISO images because it computes the inode number from the byte address
of the directory entry. Two entries = two different inode numbers.
So ISO images produced from /mnt contain two copies of vmlinuz.


Have a nice day :)

Thomas


--

Tim Kientzle

unread,
Jan 25, 2011, 12:10:01 PM1/25/11
to
Thomas,

It will be a day or two before I can dig deeply into this.

Unfortunately, it's been a while since I looked at that
part of the code in detail, but I don't recall libarchive
requiring all directories to precede all files and
I recall a bunch of test cases over the last couple of years
dealing with various kinds of empty content. So I
suspect the situation is a little less dire than
you think. ;-)

Hmmm.... Are you working with libarchive 2.8.4?
There have been a number of fixes in trunk specifically
to handle symlinks and other empty data files; maybe some
of those need to be backported.

Tim

CC: Michihiro, who has been doing a lot of work in
this part of libarchive recently.

> You received this message because you are subscribed to the Google Groups "libarchive-discuss" group.
> To post to this group, send email to libarchiv...@googlegroups.com.
> To unsubscribe from this group, send email to libarchive-disc...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/libarchive-discuss?hl=en.

Thomas Schmitt

unread,
Jan 25, 2011, 1:40:02 PM1/25/11
to
Hi,

Tim Kientzle:


> Hmmm.... Are you working with libarchive 2.8.4?

Yes. That is what Debian installed when being asked for bsdtar.

> There have been a number of fixes in trunk specifically
> to handle symlinks and other empty data files; maybe some
> of those need to be backported.

Argh ? All effort in vain ?
Well, Debian has xorriso-0.5.6 of May 2010. Six releases behind.
If the situation is similar with libarchive ... (cough).


> I don't recall libarchive requiring all directories to precede
> all files

Content block address is the key by which heap_add_entry() sorts
files into the sorted list. This list is iterated by next_entry()
in read_entries(). If a directory is encountered then its children
get inserted into the tree.
This iteration loop ends as soon as non-directory is encountered.

So, to get a complete list, all directories have to be iterated
before any non-directory comes out of next_entry().

> I recall a bunch of test cases over the last couple of years
> dealing with various kinds of empty content.

If the test images were from mkisofs or genisoimage then it is no
wonder. The ECMA-119 entries which carry Rock Ridge symbolic links
get a block address in the same range as non-empty data files.
I.e. a heap_add_entry() key that is higher than any directory key.

The ECMA-119 entries produced by libisofs for symlinks bear block
address 0. So the first link appears very early in the iteration.
The list then contains only the files from the root directory.
And more is not restored to disk.

> CC: Michihiro, who has been doing a lot of work in
> this part of libarchive recently.

The Cc: list gets longer and longer :))


Have a nice day :)

Thomas


--

Michihiro NAKAJIMA

unread,
Jan 25, 2011, 2:30:01 PM1/25/11
to
Hi,

I think it is the same thing that I fixed in libarchive trunk/r2940.
xorriso set 0 to the location of symlink files but our ISO reader
uses it to sort all entries in an ISO image to read them in
stream, and the reader expects that all directory entries appear
before any other type of entries. So that set 0 to the location
stops the reading a directory entries.

Could you try the following path against libarchive 2.8.4 ?

Index: libarchive/archive_read_support_format_iso9660.c
===================================================================
--- libarchive/archive_read_support_format_iso9660.c (revision 2945)
+++ libarchive/archive_read_support_format_iso9660.c (working copy)
@@ -1807,14 +1807,17 @@

* NOTE: Old mkisofs did not record that FILE SERIAL NUMBER
* in ISO images.
*/
- if (file->size == 0 && location >= 0)
+ if (file->size == 0 && location >= 0) {
/* If file->size is zero, its location points wrong place.
* Dot not use it for file number.
* When location has negative value, it can be used
* for file number.
*/
file->number = -1;
- else
+ /* Do not appear before any directoy entries. */
+ if (file->offset == 0)
+ file->offset = -1;
+ } else
file->number = (int64_t)(uint32_t)location;

/* Rockridge extensions overwrite information from above. */


Michihiro

2011/1/26 Tim Kientzle <t...@kientzle.com>:

Thomas Schmitt

unread,
Jan 25, 2011, 3:40:02 PM1/25/11
to
Hi,

Michihiro NAKAJIMA:


> I think it is the same thing that I fixed in libarchive trunk/r2940.

We should have Cc'ed you earlier.

> + if (file->size == 0 && location >= 0) {

> + /* Do not appear before any directoy entries. */
> + if (file->offset == 0)
> + file->offset = -1;

Now it works with debian-squeeze-di-rc2-i386-businesscard.iso after
cleaning the reserved field in the Volume Descriptor Set Terminator.

How come you know of libisofs symlink addresses but did not
stumble over the garbage in the terminator ?
(Debian bug 610781, isVDSetTerminator: "Reserved field must be 0".
That garbage was our fault, of course.)


Well, i will nevertheless complete the remedy inside libisofs and
xorriso. It would be neat if the Debian install images can be extracted
by the Debian package of bsdtar. And overall, that new block address
seems neater than the ones used up to now.

To Steve McIntyre: Upload is planned for tomorrow.
I now think you should give it a try with your image production.
The result will be more similar to genisoimage results.


Have a nice day :)

Thomas

--

Andreas Henriksson

unread,
Jan 26, 2011, 4:20:01 AM1/26/11
to
On Tue, Jan 25, 2011 at 07:36:11PM +0100, Thomas Schmitt wrote:
> Hi,
>
> Tim Kientzle:
> > Hmmm.... Are you working with libarchive 2.8.4?
>
> Yes. That is what Debian installed when being asked for bsdtar.

... and it is the latest upstream (stable) release.

>
> > There have been a number of fixes in trunk specifically
> > to handle symlinks and other empty data files; maybe some
> > of those need to be backported.
>
> Argh ? All effort in vain ?
> Well, Debian has xorriso-0.5.6 of May 2010. Six releases behind.
> If the situation is similar with libarchive ... (cough).
>

(If it's true that the released versions of libarchive are no longer supported,
then I've missed the announcement.)

Here's a list of upstream commits that I've taken the liberty to backport
and include in the debian package so far:

http://git.debian.org/?p=collab-maint/pkg-libarchive.git;a=shortlog;h=refs/heads/patches

Feel free to suggest additional suitable revisions that I should include.
(I'll start with the already mentioned revision).

... ofcourse, a new 2.8(.5) release would be even better so other
distributions also will get the suitable fixes.


--
Andreas Henriksson

Thomas Schmitt

unread,
Jan 26, 2011, 9:30:02 AM1/26/11
to
Hi,

the new development snapshot of GNU xorriso is uploaded.
http://www.gnu.org/software/xorriso/xorriso-1.0.1.tar.gz

Version timestamp 2011.01.26.133107, MD5 d90b1502f3d4c116931024497a260f83

For a test i created a new ISO image from the content of
debian-squeeze-di-rc2-i386-businesscard.iso by:

mount -o loop debian-squeeze-di-rc2-i386-businesscard.iso /mnt

dd if=debian-squeeze-di-rc2-i386-businesscard.iso \
bs=1K count=32 of=debian.sysarea

xorriso -as mkisofs -o debian-businesscard-xorriso.iso \
-allow-leading-dots -J -R \
-b isolinux/isolinux.bin -c isolinux/boot.cat \
-no-emul-boot -boot-load-size 4 -boot-info-table \
-isohybrid-mbr debian.sysarea -partition_offset 16 \
/mnt

Tests passed:

- It can be mounted by Linux. A diff -r with the mounted original
yields the plausible result:
diff: /mnt/debian: recursive directory loop
Files /mnt/isolinux/boot.cat and /mnt2/isolinux/boot.cat differ
Files /mnt/isolinux/isolinux.bin and /mnt2/isolinux/isolinux.bin differ
(The differing files contain the block address of isolinux.bin which
is not the same in both images.)

- It unpacks with libarchive 2.8.4 (without the patch by Michihiro
Nakajima) to the same file tree as does an image created by
genisoimage.

- It boots from CD on a amd64 to an install menu with a little
rocket.

- It boots from USB stick to the same menu.

So to my best knowledge it should be safe for production.
I use it for my own backups now.


Have a nice day :)

Thomas


Steve McIntyre

unread,
Jan 26, 2011, 9:50:02 AM1/26/11
to

Cool. :-)

I've just taken that and built it ready for use in the next Debian CD builds.

--
Steve McIntyre, Cambridge, UK. st...@einval.com

"Because heaters aren't purple!" -- Catherine Pitt

Thomas Schmitt

unread,
Jan 27, 2011, 3:30:02 AM1/27/11
to
Hi,

the new daily image nicely passed my tests
http://cdimage.debian.org/cdimage/daily-builds/daily/current/i386/iso-cd/debian-testing-i386-businesscard.iso
It boots with readable help texts, unpacks completely with unaltered
libarchive-2.8.4 and with patched libarchive-2.8.4.

If there are more test ideas: please do and report any suspicious effects.


Have a nice day :)

Thomas


Samuel Thibault

unread,
Jan 27, 2011, 4:50:03 AM1/27/11
to
Thomas Schmitt, le Thu 27 Jan 2011 09:24:53 +0100, a écrit :
> the new daily image nicely passed my tests
> http://cdimage.debian.org/cdimage/daily-builds/daily/current/i386/iso-cd/debian-testing-i386-businesscard.iso
> It boots with readable help texts, unpacks completely with unaltered
> libarchive-2.8.4 and with patched libarchive-2.8.4.
>
> If there are more test ideas: please do and report any suspicious effects.

It looks all good.

Samuel

0 new messages