Installer Respecting Symlinks on Non-Boot Volumes

[Gary Larizza]

Hi All,

I'm trying to debug an issue that came up in using InstaDMG, but may
not necessarily reside with the program. A little background - I'm
using a 10.6.2 machine to create an image with InstaDMG 1.6b2
(revision 261) using a 10.6 Volume License CD. I load the build train
up with the 10.6.2 combo installer and two packages: Puppet .25.4 and
Facter 1.5.7 ( https://sites.google.com/a/explanatorygap.net/puppet/
). The problem lies with the fact that Puppet needs to install files
to /usr/lib/ruby/site_ruby which is actually a symlinked folder. As
it turns out, installing the package on a non-boot volume blows away
this symlink and replaces it with a simple folder...which breaks the
program. Installing the same packages on a Boot Volume are not a
problem, as installer respects the symlinks. See the link for a
picture of the symlink-breaking action --> http://twitpic.com/17xyu7

I was able to install puppet using InstaDMG 1.5 on a 10.5.8 machine
and building a 10.5.8 image...so I'm assuming that the issue may
reside with installer in 10.6. Has anyone else run into issues with
packages respecting symlinks like this before?

-Gary

[Karl Kuehn]

On Mar 11, 2010, at 12:05 PM, Gary Larizza wrote:

The problem lies with the fact that Puppet needs to install files to /usr/lib/ruby/site_ruby which is actually a symlinked folder.  As it turns out, installing the package on a non-boot volume blows away this symlink and replaces it with a simple folder...which breaks the program.  Installing the same packages on a Boot Volume are not a problem, as installer respects the symlinks.  See the link for a picture of the symlink-breaking action --> http://twitpic.com/17xyu7

I was able to install puppet using InstaDMG 1.5 on a 10.5.8 machine and building a 10.5.8 image...so I'm assuming that the issue may reside with installer in 10.6.  Has anyone else run into issues with packages respecting symlinks like this before?

I tried to create a simple reproducible case (with the notion of filing a bug with Apple), but so far have not been able to get the bug to reproduce. As a starting point I am including my test rig. Depending on your UID you might have to run this with sudo, but the sort of it is that I have not yet been able to get to the bug as I imagine it.

The 10.5 difference is probably because we can use chroot there, and that bypasses the problem (is my guess), but to be certain about that I am going to have to be able to re-procue the bug in a striped-down setup (like this test rig).

--

Karl Kuehn

lar...@softhome.net

[Nigel Kersten]

It's possible I may have screwed up something or failed to account for
newer installers in the script I have generating the puppet and facter
packages....

If anyone sees an obvious smoking gun at:
http://github.com/reductivelabs/puppet/blob/0.25.4/conf/osx/createpackage.sh

I'd appreciate it, as I don't have any other Mac packaging gurus who
code review this :) The other supporting files are in that same
directory.

> --
> You received this message because you are subscribed to the Google Groups
> "InstaDMG-Dev" group.
> To post to this group, send email to instad...@googlegroups.com.
> To unsubscribe from this group, send email to
> instadmg-dev...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/instadmg-dev?hl=en.
>
>
>
>

[Gary Larizza Jr.]

I did some playing on a couple of machines (10.6.2 and 10.6.1 using InstaDMG 1.6b2 downloaded yesterday) and was unable to reproduce the error when installing the package via commandline with installer. I tried installing to a sparseimage, a folder, and a separate mounted drive - every time it respected the symlink. On a lark, I fired up InstaDMG, dropped the Puppet packages into the build train, and watched the /private/tmp/InstaDMGxxxx directory that's used as a staging area for the DMG file. What I found was that the symlink to /usr/lib/ruby/site_ruby is intact until either the Facter or Puppet package begin their installation...at which point /usr/lib/ruby/site_ruby gets changed into a regular directory. InstaDMG actually recognizes that permissions are wrong with this folder when it attempts to account for symlinks, but it doesn't appear to do anything about it.

I threw together a quick video here (open with Safari - it doesn't work with Chrome) --> http://www.screencast-o-matic.com/watch/c6elnL1DJ

I'm going to play more tomorrow with a mounted firewire drive that contains a base 10.6 system and see if I can reproduce this with the 'installer -pkg /path/to/the.pkg -target /Volumes/firewiredrive' command to see where the root of the problem lies.

-Gary

[Joe Block]

I'll have a look at it later today, I was planning to build a new master image at work soon anyway.

--
Don't say the sky's the limit when there are footprints on the moon.

[Gary Larizza Jr.]

Karl,
Can you try something for me?  I'm using InstaDMG 1.6b2 (revision 261).  Drop a 10.6 Installer base image, and add a couple of packages into the build train (only to keep InstaDMG Busy while you navigate it's Temporary Mount Point).  Start the instadmg.bash script, and in another terminal window navigate to /private/tmp/InstaDMG_temp_folderxx/mount_point.xxxxx/usr/lib/ruby and do an ls -l.  Then try navigating to the site_ruby folder - it gives me "No such file or directory"  - do you see the same functionality?

-Gary

--
You received this message because you are subscribed to the Google Groups "InstaDMG-Dev" group.
To post to this group, send email to instad...@googlegroups.com.
To unsubscribe from this group, send email to instadmg-dev...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/instadmg-dev?hl=en.

<softlinkTest.zip>

[Nigel Kersten]

On Thu, Mar 11, 2010 at 2:33 PM, Karl Kuehn <kuehn...@gmail.com> wrote:

So I have a theory based upon more info from Gary, but I don't do
InstaDMG at Google at all anymore (other people are, just not me) and
don't have a test rig there.

Is it that symlink exists, but it's target doesn't at the time
InstaDMG is installing? and thus it can't follow it?

I've forgotten what the term is for that "install stuff at the first
reboot after this package which requires a restart installs" stuff
Apple started introducing with a whole bunch of their system packages,
but it sounds like this directory might be one of those.

I imagine if that is the case that I'll be able to do something like
actually put the target of the symlink in my packages so that it's
guaranteed to be there on install time.

This kind of sucks though, as it means we're going to need different
packages for 10.4 and 10.{5,6}, or I create some stupid /Library/Ruby
folder on 10.4 that nothing will actually use. I don't think I can
bring myself to do the latter.

Puppet is one of those things you really want a single package for all
your clients for :(

> --
> Karl Kuehn
> lar...@softhome.net

[Nigel Kersten]

PS. If anyone has a clean 10.6 machine that hasn't ever had the Puppet
(or any other 3rd party Ruby) packages installed, can you post what
these symlnks target?

ls -lad /usr/lib/ruby
ls -lad /usr/lib/ruby/site_ruby

[Greg Neagle]

root# cd /Volumes/Snow\ Leopard/
root# ls -lad usr/lib/ruby/
drwxr-xr-x 7 root wheel 238 Sep 3 2009 usr/lib/ruby/
root# ls -lad usr/lib/ruby
lrwxrwxr-x 1 root wheel 76 Dec 7 12:01 usr/lib/ruby -> ../../
System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby
root# ls -lad usr/lib/ruby/site_ruby
lrwxrwxr-x 1 root wheel 47 Dec 7 11:57 usr/lib/ruby/site_ruby -
> ../../../../../../../../../../Library/Ruby/Site

Remember that second symlink is really:

root# ls -lad System/Library/Frameworks/Ruby.framework/Versions/
Current/usr/lib/ruby/site_ruby
lrwxrwxr-x 1 root wheel 47 Dec 7 11:57 System/Library/Frameworks/
Ruby.framework/Versions/Current/usr/lib/ruby/site_ruby -
> ../../../../../../../../../../Library/Ruby/Site

-Greg

[Jeremy Reichman]

From a Radmind-managed, non-Puppet system running 10.6.2:

$ ls -lad /usr/lib/ruby
lrwxr-xr-x 1 root wheel 76 Sep 8 2009 /usr/lib/ruby ->
../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby
$ ls -lad /usr/lib/ruby/site_ruby
lrwxr-xr-x 1 root wheel 47 Sep 8 2009 /usr/lib/ruby/site_ruby ->
../../../../../../../../../../Library/Ruby/Site

For comparison, from an unmanaged, non-Puppet system running 10.5.8:

# ls -lad /usr/lib/ruby
lrwxr-xr-x 1 root wheel 76 Oct 1 2008 /usr/lib/ruby ->
../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby
# ls -lad /usr/lib/ruby/site_ruby
lrwxr-xr-x 1 root wheel 47 Oct 1 2008 /usr/lib/ruby/site_ruby ->
../../../../../../../../../../Library/Ruby/Site

Neither have had any Ruby packages installed.

--
Jeremy

[Nigel Kersten]

Excellent. So at least 10.5 and 10.6 are the same.

Given we have a lot of people who know about packages on this list...
what do you think is the best option? Here's what I understand to be
the problem.

When you do a 10.6 install, the symlinks exist, but not the target
until the next boot.

I want to use the symlinks to install to so that the 10.4 and 10.{5,6}
packages don't need to differ, as for 10.4 this is a directory, not a
symlink.

The only options I can really think of are to have a preflight script
that creates the symlink targets *if* installing onto 10.{5,6}, but I
dislike having packages that create directories that don't show up in
the payload.

However, these are directories that the OS "owns" and will create. Is
this such a bad strategy in that case?

[Joe Block]

You've heard more than one of my rants about things installed/created that don't show up in a package listing, but as you say, the OS will create those directories eventually anyway. It's a win to create them when necessary and be able to have a unified package for all supported OS versions.

[Jeremy Reichman]

I'm seeing both the symlinks and targets exist when built with InstaDMG (v1.5rc1).

On Snow Leopard (mounted without owners):

frobozz $ cd /Volumes/InstaDMG

frobozz $ ls -lad usr/lib/rubylrwxr-xr-x  1 admin  staff  76 Oct 14 18:20 usr/lib/ruby -> ../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby

frobozz $ ls -1 System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby

1.8

gems

site_ruby

user-gems

vendor_ruby

frobozz $ ls -lad usr/lib/ruby/site_ruby

lrwxr-xr-x  1 admin  staff  47 Oct 14 18:19 usr/lib/ruby/site_ruby -> ../../../../../../../../../../Library/Ruby/Site

frobozz $ ls -1 Library/Ruby/Site

1.8

frobozz $ for RBPATH in usr/lib/ruby System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby usr/lib/ruby/site_ruby Library/Ruby/Site; do if [ -d ${RBPATH} ] ; then echo ${RBPATH} is a directory. ; fi ; if [ -h ${RBPATH} ] ; then echo ${RBPATH} is a symlink. ; fi ; done

usr/lib/ruby is a directory.

usr/lib/ruby is a symlink.

System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby is a directory.

usr/lib/ruby/site_ruby is a symlink.

Library/Ruby/Site is a directory.

On Leopard (mounted without owners):

frobozz $ cd /Volumes/InstaDMG\ 1/

frobozz $ ls -lad usr/lib/ruby

lrwxr-xr-x  1 admin  staff  76 Sep  2  2009 usr/lib/ruby -> ../../System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby

frobozz $ ls -1 System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby

1.8

gems

site_ruby

user-gems

frobozz $ ls -lad usr/lib/ruby/site_ruby 

lrwxr-xr-x  1 admin  staff  47 Sep  2  2009 usr/lib/ruby/site_ruby -> ../../../../../../../../../../Library/Ruby/Site

frobozz $ ls -1 Library/Ruby/Site

1.8

frobozz $ for RBPATH in usr/lib/ruby System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby usr/lib/ruby/site_ruby Library/Ruby/Site; do if [ -d ${RBPATH} ] ; then echo ${RBPATH} is a directory. ; fi ; if [ -h ${RBPATH} ] ; then echo ${RBPATH} is a symlink. ; fi ; done

usr/lib/ruby is a directory.

usr/lib/ruby is a symlink.

System/Library/Frameworks/Ruby.framework/Versions/Current/usr/lib/ruby is a directory.

usr/lib/ruby/site_ruby is a symlink.

Library/Ruby/Site is a directory.

[Gary Larizza Jr.]

Jeremy,
Can you change to the /usr/lib/ruby/site_ruby directory on a mounted
InstaDMG image that hasn't been booted yet? The symlink is created,
and points correctly to /Library/Ruby/Site, but navigating to the
directory produces an error...at least until the image has been booted
once (presumably because the link was created before the Target
Directory existed).

-Gary

[Karl Kuehn]

I have found the problem that is causing the bugs in installing Puppet in a InstaDMG workflow, and it is not a bug in the Puppet installer, but rather a bug in Apple's setup. We have all been chasing around the symlinks, and it turns out one of them is wrong, so this is a bug on Apple's part. The problem child is:

/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/site_ruby

(note: the 1.8 is usually done as "Current", but this symlink resolves to "1.8" in the same directory on 10.6)

This resolves to:

../../../../../../../../../../Library/Ruby/Site

If you count the back-references you will get 10, but there are only 9 levels in the absolute path. When you are booted the extra level gets swallowed as a non-fatal (and non warning) error (try: "cd /; cd.."). However, if the filesystem is mounted anywhere other than root (as it is in InstaDMG) this will cause a problem since you can ../ one more directory out, but then there is no /Library/Ruby/Site in that outer folder.

So this is all a bug on Apple's part in the symlinks, and it looks like it has been a problem since 10.5 (I just checked . I am going to report it right now. But the Puppet package is probably going to have to work around this for now by not trying to have the package follow the torturous list of softlinks, but install stuff straight to /Library/Ruby/Site.

I just submitted this as Radar number 7758836, but thinking it through I doubt that Apple will go back and fix 10.5 at this point, so I think that we are stuck working around it for installers.

And yes, this would be fixed if I could chroot the installer on 10.6. And it should work with the chroot'ed installer InstaDMG uses on 10.5.

--
Karl Kuehn
lar...@softhome.net

[Nigel Kersten]

On Tue, Mar 16, 2010 at 10:57 AM, Karl Kuehn <kuehn...@gmail.com> wrote:
>        I have found the problem that is causing the bugs in installing Puppet in a InstaDMG workflow, and it is not a bug in the Puppet installer, but rather a bug in Apple's setup. We have all been chasing around the symlinks, and it turns out one of them is wrong, so this is a bug on Apple's part. The problem child is:
>
> /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/site_ruby
>
>        (note: the 1.8 is usually done as "Current", but this symlink resolves to "1.8" in the same directory on 10.6)
>
>        This resolves to:
>
> ../../../../../../../../../../Library/Ruby/Site
>
>        If you count the back-references you will get 10, but there are only 9 levels in the absolute path. When you are booted the extra level gets swallowed as a non-fatal (and non warning) error (try: "cd /; cd.."). However, if the filesystem is mounted anywhere other than root (as it is in InstaDMG) this will cause a problem since you can ../ one more directory out, but then there is no /Library/Ruby/Site in that outer folder.
>
>        So this is all a bug on Apple's part in the symlinks, and it looks like it has been a problem since 10.5 (I just checked . I am going to report it right now. But the Puppet package is probably going to have to work around this for now by not trying to have the package follow the torturous list of softlinks, but install stuff straight to /Library/Ruby/Site.

Well done Karl. I really should have thought of actually counting the
damn levels :)

This brings me back to having to produce different packages for 10.4
and 10.5+ though :(

What do you all think of the option of fixing the symlink in a
preinstall if on 10.5/6 ?

>
>        I just submitted this as Radar number 7758836, but thinking it through I doubt that Apple will go back and fix 10.5 at this point, so I think that we are stuck working around it for installers.
>
>        And yes, this would be fixed if I could chroot the installer on 10.6. And it should work with the chroot'ed installer InstaDMG uses on 10.5.
>
> --
>        Karl Kuehn
>                lar...@softhome.net
>