ACL/Mac OS X problems
Jesse Peterson
TOC than what is below? If not then where are ACLs stored? BTW this
is compiled from the pkgsrc-wip collection.
Thanks!
- Jesse
% uname -a
Darwin localhost 8.8.1 Darwin Kernel Version 8.8.1: Mon Sep 25
19:42:00 PDT 2006; root:xnu-792.13.8.obj~1/RELEASE_I386 i386 i386
% cd /
% sudo fsaclctl -p / -e
% touch foo
% chmod +a 'root allow read' foo
% chmod +a 'root allow write' foo
% ls -le foo
-rw-r--r-- + 1 jesse admin 0 Jan 17 09:41 foo
0: user:root allow read,write
% xar -v -c -f foo.xar foo
foo
% xar --dump-toc=- -f foo.xar
<?xml version="1.0" encoding="UTF-8"?>
<xar>
<toc>
<checksum style="sha1">
<size>20</size>
<offset>0</offset>
</checksum>
<file id="1">
<name>foo</name>
<type>file</type>
<mode>0644</mode>
<uid>1085</uid>
<user>jesse</user>
<gid>80</gid>
<group>admin</group>
<atime>2007-01-17T17:41:51Z</atime>
<mtime>2007-01-17T17:41:51Z</mtime>
<ctime>2007-01-17T17:53:06Z</ctime>
</file>
</toc>
</xar>
Rob Braun
>
> Should ACLs be showing up in the TOC? Should I expect any different
> TOC than what is below? If not then where are ACLs stored? BTW this
> is compiled from the pkgsrc-wip collection.
Can you verify what options your version of xar was built with,
which build of osx you have, and what version of xar you're using?
Was acl support detected and built in when you built xar?
Thanks,
Rob
Jesse Peterson
config file. This is Mac OS X 10.4.8/Intel (uname included in last
message).
[...]
checking sys/acl.h usability... yes
checking sys/acl.h presence... yes
checking for sys/acl.h... yes
checking ext2fs/ext2_fs.h usability... no
checking ext2fs/ext2_fs.h presence... no
checking for ext2fs/ext2_fs.h... no
checking sys/statfs.h usability... no
checking sys/statfs.h presence... no
checking for sys/statfs.h... no
checking sys/xattr.h usability... yes
checking sys/xattr.h presence... yes
checking for sys/xattr.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/extattr.h usability... no
checking sys/extattr.h presence... no
checking for sys/extattr.h... no
checking libutil.h usability... no
checking libutil.h presence... no
checking for libutil.h... no
checking for lgetxattr... no
checking for lsetxattr... no
checking for getxattr... yes
checking for setxattr... yes
checking for getattrlist... yes
checking for setattrlist... yes
checking for uid_t... yes
checking size of uid_t... 4
checking for gid_t... yes
checking size of gid_t... 4
checking for ino_t... yes
checking size of ino_t... 4
checking for dev_t... yes
checking size of dev_t... 4
checking for acl_get_file in -lacl... no
[...]
Is that helpful?
Rob Braun
>
> Is that helpful?
What version of xar are you using? I don't use or develop for pkgsrc,
so does this work with the stock version of xar? Knowing if the
top of tree in subversion works would be helpful too.
Rob
Jesse Peterson
and svn and report back.
Thanks,
- Jesse
Jesse Peterson
/% ~/xar/xar-1.4/src/xar --dump-toc=- -f foo.xar
<?xml version="1.0" encoding="UTF-8"?>
<xar>
<toc>
<checksum style="sha1">
<size>20</size>
<offset>0</offset>
</checksum>
<file id="1">
<name>foo</name>
<type>file</type>
<mode>0644</mode>
<uid>1085</uid>
<user>jesse</user>
<gid>80</gid>
<group>admin</group>
<atime>2007-01-19T04:13:29Z</atime>
<mtime>2007-01-19T04:12:14Z</mtime>
<ctime>2007-01-19T04:13:10Z</ctime>
</file>
</toc>
</xar>
and...
% ~/xar/trunk/src/xar --dump-toc=- -f foo.xar
<?xml version="1.0" encoding="UTF-8"?>
<xar>
<toc>
<checksum style="sha1">
<size>20</size>
<offset>0</offset>
</checksum>
<file id="1">
<name>foo</name>
<type>file</type>
<mode>0644</mode>
<uid>1085</uid>
<user>jesse</user>
<gid>80</gid>
<group>admin</group>
<atime>2007-01-19T04:13:43Z</atime>
<mtime>2007-01-19T04:12:14Z</mtime>
<ctime>2007-01-19T04:13:10Z</ctime>
<data>
<extracted-checksum
style="SHA1">da39a3ee5e6b4b0d3255bfef95601890afd80709</extracted-checksum>
<archived-checksum
style="SHA1">9a39f2dd186749fea11b84e53cfa77194eed88d3</archived-checksum>
<encoding style="application/x-gzip"/>
</data>
</file>
</toc>
</xar>
for reference:
% chmod +a 'root allow write' foo
% ls -le foo
-rw-r--r-- + 1 jcp admin 0 Jan 18 20:12 foo
0: user:root allow write
All archives created like:
% [path-to-specific]xar -cvf foo.xar foo
Any debug steps? Thanks for your time!
- Jesse
Jesse Peterson
for my file in xar-1.4/lib/stat.c:130 which would seem to signify that,
well, there is no ACL entry for the file ;). However, acl_get_file
returns non-null which presumably would indicate that there is an ACL
entry on the file.
In:
http://darwinsource.opendarwin.org/10.4.2/file_cmds-116.9/ls/print.c
on lines ~ 354 and the function printacl() (lines ~ 252 - 318) appears
to being acl_get_file and acl_get_entry in a very similar way. Which of
course makes it interesting that "ls -le" prints the ACLs yet the above
is true. Thoughts?
Rob Braun
ACL support in 10.4 is very sketchy. With the initial release of 10.4.0
the acl header files did not match the actual implementation. I also
encountered cases that caused kernel panics. I filed bugs on this when
xar acl support was implemented, more than a year ago. Since then, I
have not had meaningful access to an OS X system, so I don't know if
later software updates and xcode releases have resolved or worsened
the ACL problems.
Rob
Jesse Peterson
> the acl header files did not match the actual implementation. I also
> encountered cases that caused kernel panics. I filed bugs on this when
> xar acl support was implemented, more than a year ago. Since then, I
> have not had meaningful access to an OS X system, so I don't know if
> later software updates and xcode releases have resolved or worsened
> the ACL problems.
Joy. Thanks, Apple ;).
How about implementation/man page discrepencies? Reading the source
for "ls" again I noticed that the for-loop for for getting the ACLs was
checking for a 0 from acl_get_entry(). This is in direct contraction to
the acl_get(3) man page and indeed, the below patch gives me ACLs in
xar:
--- xar-1.4/lib/stat.c.orig 2006-01-12 19:55:21.000000000 -0800
+++ xar-1.4/lib/stat.c 2007-01-20 16:53:04.000000000 -0800
@@ -127,7 +127,7 @@
acl_entry_t e;
/* If the acl is empty, or not valid, skip it */
- if( acl_get_entry(a, ACL_FIRST_ENTRY, &e) != 1 )
+ if( acl_get_entry(a, ACL_FIRST_ENTRY, &e) != 0 )
goto NEXT;
t = acl_to_text(a, NULL);
@@ -145,7 +145,7 @@
acl_entry_t e;
/* If the acl is empty, or not valid, skip it */
- if( acl_get_entry(a, ACL_FIRST_ENTRY, &e) != 1 )
+ if( acl_get_entry(a, ACL_FIRST_ENTRY, &e) != 0 )
goto DONE;
t = acl_to_text(a, NULL);
This patch gives me these ACL entries in the TOC for my example files:
<acl>
<default>!#acl 1
user:FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000:root:0:allow:write
</default>
<access>!#acl 1
user:FFFFEEEE-DDDD-CCCC-BBBB-AAAA00000000:root:0:allow:write
</access>
</acl>
I successfully extracted this xar archive to get my original ACLs back,
too. Is it worth a conditional check in the xar source for this OS to
check for a different return value of acl_get_entry()?
Rob Braun
>
> How about implementation/man page discrepencies? Reading the source
> for "ls" again I noticed that the for-loop for for getting the ACLs was
> checking for a 0 from acl_get_entry(). This is in direct contraction to
> the acl_get(3) man page and indeed, the below patch gives me ACLs in
> xar:
sigh. That means it changed at some point in the past, because I
know acls worked on early 10.4's on ppc with the existing code.
Although OSX has improved some in the past, it is still pretty
unusual to have a man page match reality.
My inclination is to apply this patch and encourage anyone that
encounters problems to upgrade their 10.4 system to the latest.
And hope it doesn't change again. I don't think this is worth
an autoconf test to see if a 0 or 1 return value is what we're
looking for.
Thanks,
Rob
Jesse Peterson
> encounters problems to upgrade their 10.4 system to the latest.
> And hope it doesn't change again. I don't think this is worth
> an autoconf test to see if a 0 or 1 return value is what we're
> looking for.
The Mac OS X man page for acl_get_entry(3) says that it is a POSIX.1e
call. Do you think there would be any other platforms that might be
effected if this patch were applied and Mac OS X is the one in error?
I'm specifically thinking of FreeBSD here (which, I'm betting, the man
page itself was stolen from). If it were important I might be able to
test on a FreeBSD machine.
Thanks,
- Jesse
Jesse Peterson
> know acls worked on early 10.4's on ppc with the existing code.
Interesting. I pulled this out of
http://www.opensource.apple.com/darwinsource/10.4/file_cmds-116.9/ls/print.c
(around line 266)
for (index = 0;
acl_get_entry(acl, entry == NULL ? ACL_FIRST_ENTRY :
ACL_NEXT_ENTRY, &entry) == 0;
index++) {
I also noticed that the only change of the file_cmds project version
number was between 10.4.6 and 10.4.7 (going from 116.9 to 116.10). An
since I made that patch off of the 10.4.2 sources (116.9) it is
unlikely that the code had changed. In fact I just checked and the
10.4.8 116.10 ls/print.c source is exactly the same as above. FWIW.
Thanks,
- Jesse
Rob Braun
>
> I also noticed that the only change of the file_cmds project version
> number was between 10.4.6 and 10.4.7 (going from 116.9 to 116.10). An
> since I made that patch off of the 10.4.2 sources (116.9) it is
> unlikely that the code had changed. In fact I just checked and the
> 10.4.8 116.10 ls/print.c source is exactly the same as above. FWIW.
I went back and checked over my history for xar and acl/ea use.
It seems I did almost all of my development & testing on linux
(specifically, redhat derivatives) and freebsd. Running a quick
test of acls and eas on a fedora system I had handy, they seem to
work as is.
A lot of the Mac OS X EA functionality is "odd", in the sense that
they have the resource fork treated "special", they use a very different
naming scheme (com.apple, vs user. or system.), and the apis are
named similar, but have very different semantics.
Mac OS X's ACL functionality, although announced as a feature of 10.4,
are really not there for third party integration yet. I'd expect this
to improve in 10.5. However, bugs aside, some of the design of Mac
OS X acls is disconcerting. There is a posix specification for the
serialization of acls, which Mac OS X does not conform to, nor is
there a way to convert between them.
My general feeling is to abandon 10.4 for acl support, and wait until it
is actually usable in 10.5. And understand that it will have to be highly
conditionalized due to the 'enhanced' interface.
Rob
Jesse Peterson
On Jan 28, 10:19 am, Rob Braun <bbr...@synack.net> wrote:
> My general feeling is to abandon 10.4 for acl support, and wait until it
> is actually usable in 10.5. And understand that it will have to be highly
> conditionalized due to the 'enhanced' interface.
Hmm. What an unfortunately situation. I intuit that dropping support
sort of goes against
I sort of feel like xar should support this case. For even though the
code may be unpretty here or there to support the occasional platform-
specific "hack," it is the eXtensible ARchiver. Arbitraty metadata
capture is sort of its job. I understand that it might not make sense
to support every little peice of arcane or proprietary peice of
metadata, but it seems like a good idea to make a best-effort case for
those that make sense.
Alternatively I suggest a repository (probably a wiki page?) of cases
like these where platform (specific or not) metadata to be (or not to
be!) archived can be documented with an explanation of what, why, how,
and links to bug reports, etc relating to the data. If a platform is
so broken so as not support reliable metadata capture, then that is
worthy of note, to at least let people know what to expect.
I'd like to hope that xar could be data-archival quality at some point
whereby an entire filesystem and all associated metadata could be
captured for exact filesystem duplication/restoration. Its probably
clear I'm speaking to xar as a systems tool, and that may not be where
xar wants to be - especially with its API and such - though I
personally see them as hand in hand. Obviously an unrelated spiel
here, but I'd like xar to be depended on by its users and known for
what it does best - being an extensible archiver :).
Rob Braun
>
> I sort of feel like xar should support this case. For even though the
> code may be unpretty here or there to support the occasional platform-
> specific "hack," it is the eXtensible ARchiver. Arbitraty metadata
> capture is sort of its job. I understand that it might not make sense
> to support every little peice of arcane or proprietary peice of
> metadata, but it seems like a good idea to make a best-effort case for
> those that make sense.
I can look into it. The semantics of the apis are probably going
to change in 10.5, breaking anything we do, and be left with support
for half-implemented ACL support, and a way to differentiate between
10.4 and 10.5. My understanding is ACLs aren't obeyed on 10.4 without
a kernel flag being set.
> Alternatively I suggest a repository (probably a wiki page?) of cases
> like these where platform (specific or not) metadata to be (or not to
> be!) archived can be documented with an explanation of what, why, how,
> and links to bug reports, etc relating to the data. If a platform is
> so broken so as not support reliable metadata capture, then that is
> worthy of note, to at least let people know what to expect.
This is probably a good idea regardless, if for no other reason than
as a todo list. There are still lots of systems we don't support,
such as Tru64, which has its own interesting metadata.
> I'd like to hope that xar could be data-archival quality at some point
> whereby an entire filesystem and all associated metadata could be
> captured for exact filesystem duplication/restoration. Its probably
> clear I'm speaking to xar as a systems tool, and that may not be where
> xar wants to be - especially with its API and such - though I
> personally see them as hand in hand. Obviously an unrelated spiel
> here, but I'd like xar to be depended on by its users and known for
> what it does best - being an extensible archiver :).
You bring up a point that I've thought about, but probably haven't
articulated before. There is volume specific information that is
not associated with files, that xar doesn't currently have support
for. Differentiating between archiving a directory and a volume
would be interesting to think about further. If we knew we were
archiving a volume, we could include the volume specific information
in a new container at the root of the <toc>.
Rob
Jesse Peterson
On Jan 28, 10:19 am, Rob Braun <bbr...@synack.net> wrote:
> There is a posix specification for the
> serialization of acls, which Mac OS X does not conform to, nor is
> there a way to convert between them.
So I looked into this a little and found that said POSIX spec, "1e"
that all of the implementations are defined against was never actually
a standard. In fact it never made it past a draft status. Further it
was actually withdrawn altogether.
This is not an excuse for Apple, however, and I think having at least
SOME spec in the community is worth adhering to. In fact I'm rather
stupefied at the fact the POSIX.1e spec has in the "long text" ACL
representation specification extensibility BUILT-IN in the form of
implementation-defined fields! Why on earth they chose to mess with
the specified behaviour and not color in between the lines (especially
when the lines were drawn extra thick for them!) is beyond me.
I think I feel a little bit of your pain now...
- Jesse