Trusted package not installing via munki... ?

[Heig Gregorian]

So, here's MacPorts 2.1.2 for Mountain Lion.  Installs without issue via GUI, without issue via command-line 'installer -pkg...-tgt /'

Here's what I'm getting in munki:

Installing MacPorts (1 of 1)...

    suppress_bundle_relocation: False

    Mounting disk image MacPorts-2.1.2-10.8-MountainLion.dmg...

    Install of MacPorts failed with return code 1...

ERROR: ------------------------------------------------------------------------------

ERROR: installer: Package name is MacPorts

ERROR: installer: Certificate used to sign package is not trusted. Use -allowUntrusted to override.

ERROR: ------------------------------------------------------------------------------

Finishing...

    Getting info on currently installed applications...

    Excluding /mcx (flags True, nfs False)

    Performing postflight tasks...

    postflight stdout: postflight logged for YPCMC10010

Done.

Here's the signature output from pkgutil:

$ pkgutil --check-signature ~/Desktop/MacPorts-2.1.2-10.8-MountainLion.pkg 

Package "MacPorts-2.1.2-10.8-MountainLion.pkg":

   Status: signed by a certificate trusted by Mac OS X

   Certificate Chain:

    1. Developer ID Installer: Robert Mercer

       SHA1 fingerprint: 36 B6 D7 DA A6 7E 94 93 B8 13 96 0C CB 21 0B 99 A8 A1 C9 6E

       -----------------------------------------------------------------------------

    2. Developer ID Certification Authority

       SHA1 fingerprint: 3B 16 6C 3B 7D C4 B7 51 C9 FE 2A FA B9 13 56 41 E3 88 E1 86

       -----------------------------------------------------------------------------

    3. Apple Root CA

       SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60

[Heig Gregorian]

Another detail.

pkgutil --expand and --flatten to remove the cert on the package does allow it to install correctly via munki.

[Greg Neagle]

On Jul 26, 2012, at 1:52 PM, Heig Gregorian wrote:

Another detail.

pkgutil --expand and --flatten to remove the cert on the package does allow it to install correctly via munki.

Sure, because that removes the signing/certificate. It becomes an unsigned package at that point.

-Greg

[Greg Neagle]

I may not have time to look at this in detail for some time.

If you want to start tearing it apart, start adding more and more of Munki's layers to see where it fails.

You'll want to look at the munkilib/installer.py and munkilib/launchd.py for functions to borrow/ steal. Example:

cd munkilib
python
import installer
installer.install('/path/to/MacPorts-2.1.2-10.8-MountainLion.pkg')

-Greg

[Heig Gregorian]

Yes, I know :)

But why the discrepancy between the munki installation and the GUI / command-line installation in the first place?

--Heig

[Greg Neagle]

On Jul 26, 2012, at 2:09 PM, Heig Gregorian wrote:

Yes, I know :)

But why the discrepancy between the munki installation and the GUI / command-line installation in the first place?

I don't know -- go find out!

-Greg

[Heig Gregorian]

<phhhh>...FINE.

The naîve example you provided is broken too...I'll dig a bit.

--Heig

[Greg Neagle]

Teach a man to fish...

...and he can catch fish for you.

-Greg

[Heig Gregorian]

And here's the fish...just don't know exactly kind it is:

Stripped it down to something to do with the environment variable "HOME"

This works:

env_vars = os.environ.copy()

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

This doesn't:

env_vars = os.environ.copy()

env_vars['HOME'] = userinfo.pw_dir

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

So, env_vars['HOME'] defaults to '/var/root' and setting it to a real, other than root home directory makes things work, but what does that have to do with a certificate?

I've gone ahead with adding the following to pkginfo: (this works)

<key>installer_environment</key>

<dict>

<key>USER</key>

<string>CURRENT_CONSOLE_USER</string>

</dict>

--Heig

[Greg Neagle]

Thank you for fishing. Comments below.

On Jul 26, 2012, at 2:48 PM, Heig Gregorian wrote:

And here's the fish...just don't know exactly kind it is:

Stripped it down to something to do with the environment variable "HOME"

This works:

env_vars = os.environ.copy()

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

This doesn't:

env_vars = os.environ.copy()

env_vars['HOME'] = userinfo.pw_dir

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

So, env_vars['HOME'] defaults to '/var/root' and setting it to a real, other than root home directory makes things work, but what does that have to do with a certificate?

I've gone ahead with adding the following to pkginfo: (this works)

<key>installer_environment</key>

<dict>

<key>USER</key>

<string>CURRENT_CONSOLE_USER</string>

</dict>

That, of course, will not do the right thing if no user is logged in at the console -- in that case, HOME will once again be /var/root.

Setting USER and HOME in the installer_environment was added at some point last year to work around common failures in package scripts that were poorly tested and assumed that packages would only be installed when a GUI user was logged in.

It was suggested that there would be no harm in setting these values, and the benefit would be that poorly-written/poorly-tested package scripts would now magically work with Munki.

Hah.

See this discussion: https://groups.google.com/forum/?fromgroups#!searchin/munki-dev/installer$20environment/munki-dev/4-MQweQOnmU/A5gytbX5SYsJ

And this: https://groups.google.com/forum/?fromgroups#!searchin/munki-dev/installer$20environment/munki-dev/ZBjBhxBjjx0/LtqRZIhP82MJ

I notice you (Heig) were involved in that particular thread...

So: how should this be addressed?

[Heig Gregorian]

On Jul 26, 2012, at 3:18 PM, Greg Neagle wrote:

Thank you for fishing. Comments below.

On Jul 26, 2012, at 2:48 PM, Heig Gregorian wrote:

And here's the fish...just don't know exactly kind it is:

Stripped it down to something to do with the environment variable "HOME"

This works:

env_vars = os.environ.copy()

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

This doesn't:

env_vars = os.environ.copy()

env_vars['HOME'] = userinfo.pw_dir

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

So, env_vars['HOME'] defaults to '/var/root' and setting it to a real, other than root home directory makes things work, but what does that have to do with a certificate?

I've gone ahead with adding the following to pkginfo: (this works)

<key>installer_environment</key>

<dict>

<key>USER</key>

<string>CURRENT_CONSOLE_USER</string>

</dict>

That, of course, will not do the right thing if no user is logged in at the console -- in that case, HOME will once again be /var/root.

Yup, and we knew that getting into it.  In the case of this package, it will perpetually fail, which I'm totally fine with believe it or not.

Setting USER and HOME in the installer_environment was added at some point last year to work around common failures in package scripts that were poorly tested and assumed that packages would only be installed when a GUI user was logged in.

It was suggested that there would be no harm in setting these values, and the benefit would be that poorly-written/poorly-tested package scripts would now magically work with Munki.

Hah.

See this discussion: https://groups.google.com/forum/?fromgroups#!searchin/munki-dev/installer$20environment/munki-dev/4-MQweQOnmU/A5gytbX5SYsJ

And this: https://groups.google.com/forum/?fromgroups#!searchin/munki-dev/installer$20environment/munki-dev/ZBjBhxBjjx0/LtqRZIhP82MJ

I notice you (Heig) were involved in that particular thread...

So: how should this be addressed?

Yes!  I do recall being involved in this (and at some point I think I even contributed a small snippet of code).  I still support the ability for the key/value USER:CURRENT_CONSOLE_USER as it does provide a viable option in select cases.

However, what's the connection between a package's cert and, in this case, the environment variable 'HOME'?  Remember, when I stripped the cert, everything was peachy, even with 'HOME' set to '/var/root'.  I fished, and now I have more questions than I started with!

--Heig

[Greg Neagle]

On Jul 26, 2012, at 3:37 PM, Heig Gregorian wrote:

However, what's the connection between a package's cert and, in this case, the environment variable 'HOME'?  Remember, when I stripped the cert, everything was peachy, even with 'HOME' set to '/var/root'.  I fished, and now I have more questions than I started with!

I don't know -- that's a question for Apple, or at least someone who understands all this better than I.

Obviously when the environment's home is '/var/root' it affects verification of package signatures. Why? Maybe it's a bug!

-Greg

[Rob Middleton]

Heig,

What happens if you set HOME to:

/private/var/root

?

(ie: perhaps some odd bug with a symlink in the home directory path. I can't otherwise think how the root home is intrinsically different)

Cheers,

Rob.

[Heig Gregorian]

No dice.

[Gregory Neagle]

What happens if you set HOME to an empty string?

I'd ask about unsettling HOME, but the current Munki code does not support that. 

Sent from my iPhone

[Heig Gregorian]

Empty string, nope.

I've tested invalid paths, no success

I've tested valid non-home directory paths, no success...

So, really, it looks like it needs to be a valid home directory path that's not /var/root...

--Heig

[Gregory Neagle]

Or $HOME not set at all, correct?

Sent from my iPhone

[Heig Gregorian]

Correct, empty HOME and no HOME produce the same failures.

--Heig

[Gregory Neagle]

Wait. But when the installer environment was not explicitly set, it succeeded. So there is more to this than the value of $HOME...

Sent from my iPhone

[Heig Gregorian]

Ummmm...that's sort of correct...only because HOME retained the value from whatever os.environ.copy() contained...which would be the current console user.

Immediately after that, we're essentially setting it to '/var/root'

--Heig

[Greg Neagle]

So what I'd like you to test is a code change that allows you to _unset_ HOME and USER; IOW, not use the defaults, but actually unset/undefine those.

-Greg

[Rob Middleton]

For that same package on *Lion* pkgutil --check-signature appears to validate with root as home:

robair:var robadmin$ sudo bash
bash-3.2# su root
sh-3.2# echo $HOME
/var/root
sh-3.2# pkgutil --check-signature MacPorts-2.1.2-10.8-MountainLion.pkg

Package "MacPorts-2.1.2-10.8-MountainLion.pkg":
Status: signed by a certificate trusted by Mac OS X
Certificate Chain:
1. Developer ID Installer: Robert Mercer

2. Developer ID Certification Authority

3. Apple Root CA

Suggests a 10.8 bug (I don't have 10.8 installed .. I'd run the same test there, if it fails that starts to look at why the signature check is failing without getting into other complexities of the install process).

Rob.

[Heig Gregorian]

By unset do you mean, revert whatever changes were made by the following lines?

installer.py 140:    env_vars['USER'] = userinfo.pw_name

installer.py 141:    env_vars['HOME'] = userinfo.pw_dir

--Heig

[Greg Neagle]

Yes.

if something:

del env_vars['USER'] 

del env_vars['HOME'] 

or some other change that results in a code path where env_vars['USER'] and env_vars['HOME']  are undefined.

-Greg

[Heig Gregorian]

You're a genius:

YPCMC10010:~ hgregorian$ sudo bash
bash-3.2# su root
sh-3.2# pkgutil --check-signature /Users/hgregorian/Desktop/MacPorts-2.1.2-10.8-MountainLion.pkg
Package "MacPorts-2.1.2-10.8-MountainLion.pkg":
Status: invalid signature
sh-3.2#

--Heig

[Heig Gregorian]

I'm sorry, but I suppose I've said something confusing somewhere...

del env_vars['HOME'] does NOT work.  As far as I can tell $HOME needs to be set to a valid user home that's not root.

--Heig

[Greg Neagle]

Now you have a simple, easy to reproduce and document, bug you can file with Apple.

-Greg

[Heig Gregorian]

Yah....crazy...

--Heig

[Greg Neagle]

I'll have to double-check on this, but I think the X11 package here: http://xquartz.macosforge.org/downloads/SL/XQuartz-2.7.2.dmg may exhibit the same issue.

I didn't connect this with your issue, Heig, because this package did give me an "Status: invalid signature" when I used pkgutil --check-signature -- but I had sshed in as root.

-Greg

On Jul 26, 2012, at 5:27 PM, Heig Gregorian wrote:

[Greg Neagle]

On Jul 26, 2012, at 5:29 PM, Heig Gregorian wrote:

I'm sorry, but I suppose I've said something confusing somewhere...

del env_vars['HOME'] does NOT work.  As far as I can tell $HOME needs to be set to a valid user home that's not root.

Quoting you from earlier:

This works:

env_vars = os.environ.copy()

SO WHAT IS env_vars['HOME']  SET TO HERE?

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

This doesn't:

env_vars = os.environ.copy()

env_vars['HOME'] = userinfo.pw_dir

job = launchd.Job(cmd, environment_vars=env_vars)

job.start()

[Rob Middleton]

Yep! After the bug report is in we could look to see if there is a workaround.

Probably a DTrace watching file opens of "pkgutil --check-signature" in both root & not root. It may be that Apple is looking for a little preference file or similar that doesn't exist in an account that never gets a GUI login. Or the shortcut to that being a possibility -- enable the root account, login as root to the GUI (is that possible?) and see if it works then (either while as the GUI root or subsequently logged back in as a normal account).

Rob.

[Greg Neagle]

Yup. Exact same issue. I'll file a separate bug.

Lion machine:

sh-3.2# whoami
root
sh-3.2# sw_vers
ProductName: Mac OS X
ProductVersion: 10.7.4
BuildVersion: 11E53
sh-3.2# pkgutil --check-signature XQuartz.pkg
Package "XQuartz.pkg":

Status: signed by a certificate trusted by Mac OS X
Certificate Chain:

1. Developer ID Installer: Apple Inc. - XQuartz

2. Developer ID Certification Authority
3. Apple Root CA

sh-3.2#

Mountain Lion machine:

sh-3.2# whoami
root
sh-3.2# sw_vers
ProductName: Mac OS X
ProductVersion: 10.8
BuildVersion: 12A269
sh-3.2# pkgutil --check-signature XQuartz.pkg
Package "XQuartz.pkg":
Status: invalid signature
sh-3.2#

-Greg

[Heig Gregorian]

Sent from my iPhone

On Jul 26, 2012, at 5:34 PM, Greg Neagle <gregn...@mac.com> wrote:

On Jul 26, 2012, at 5:29 PM, Heig Gregorian wrote:

I'm sorry, but I suppose I've said something confusing somewhere...

del env_vars['HOME'] does NOT work.  As far as I can tell $HOME needs to be set to a valid user home that's not root.

Quoting you from earlier:

This works:

env_vars = os.environ.copy()

SO WHAT IS env_vars['HOME']  SET TO HERE?

/Users/hgregorian

Probably because I entered the python environment like this:

$ sudo python

When I should've done something like this (I think):

$ sudo bash

$ su

$ python

Or maybe even:

$ sudo su

$ python

Sorry for the confusion!

[Greg Neagle]

Bug ID# 11970185

[Heig Gregorian]

Wait wait wait wait...

I've got some weirdness going on...check this out:

#### FRESH 10.8 Image ####

sh-3.2# whoami

root

sh-3.2# sw_vers

ProductName: Mac OS X

ProductVersion: 10.8

BuildVersion: 12A269

sh-3.2# pkgutil --check-signature MacPorts-2.1.2-10.8-MountainLion.pkg 

Package "MacPorts-2.1.2-10.8-MountainLion.pkg":

   Status: invalid signature

sh-3.2# shasum -a 256 MacPorts-2.1.2-10.8-MountainLion.pkg 

a3c389696524997c1802c9232a1472046efdbd02a3031bb3125d104dc514f9ad  MacPorts-2.1.2-10.8-MountainLion.pkg

#### Upgraded from 10.7.4 ####

sh-3.2# whoami

root

sh-3.2# sw_vers

ProductName: Mac OS X

ProductVersion: 10.8

BuildVersion: 12A269

sh-3.2# pkgutil --check-signature MacPorts-2.1.2-10.8-MountainLion.pkg 

Package "MacPorts-2.1.2-10.8-MountainLion.pkg":

   Status: signed by a certificate trusted by Mac OS X

   Certificate Chain:

    1. Developer ID Installer: Robert Mercer

       SHA1 fingerprint: 36 B6 D7 DA A6 7E 94 93 B8 13 96 0C CB 21 0B 99 A8 A1 C9 6E

       -----------------------------------------------------------------------------

    2. Developer ID Certification Authority

       SHA1 fingerprint: 3B 16 6C 3B 7D C4 B7 51 C9 FE 2A FA B9 13 56 41 E3 88 E1 86

       -----------------------------------------------------------------------------

    3. Apple Root CA

       SHA1 fingerprint: 61 1E 5B 66 2C 59 3A 08 FF 58 D1 4A E2 24 52 D1 98 DF 6C 60

sh-3.2# shasum -a 256 MacPorts-2.1.2-10.8-MountainLion.pkg 

a3c389696524997c1802c9232a1472046efdbd02a3031bb3125d104dc514f9ad  MacPorts-2.1.2-10.8-MountainLion.pkg

[Rob Middleton]

That is really good news. It means we could implement a workaround while we wait for Apple.

There will be some file missing from the default raw root home directory that will make this process happy. (as per previous email of logging into the GUI as root to see if that "fixes" it -- this answers that question in the positive)

Could you run a:

sudo find /var/root

on a raw 10.8 -- what files have we got?

Maybe it is even missing an empty Library/Preferences folder ?

Cheers,

Rob.

[Heig Gregorian]

Read my mind...

Working on it...I'll let you know what my findings are in a bit.

Sent from my iPhone

[Heig Gregorian]

So...going at this from the 10.8 machine that works.

Moved Library to Library.orig ---- BROKEN!!!

I'll keep digging...

--Heig

[Heig Gregorian]

/var/root/Library/Preferences/com.apple.security.plist

But before you get excited...on the working system, its contents are populated with Symantec keychain stuff.

Copying /Library/Preferences/com.apple.security.plist does NOT remedy the issue...

The non-working system, of course, has no /var/root/Library/Preferences/com.apple.security.plist

--Heig

On Jul 26, 2012, at 6:42 PM, Rob Middleton wrote:

[Rob Middleton]

What about:

<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">

<plist version="1.0">

<dict>

<key>DLDBSearchList</key>

<array>

</array>

</dict>

</plist>

Lets give it a null array to walk through rather than an absent array to crash on?

Rob.

[Heig Gregorian]

Nope...

I'm beginning to think this has something to do with the System.keychain...

--Heig

[Greg Neagle]

I thought about pointing out there could be more differences between a "fresh" 10.8 install and an upgrade install than the contents of /var/root, but you two were having so much fun...

-Greg

[Heig Gregorian]

Here's some more fun...I wrote a haiku:

You're so smart, wise guy...

Do you have an idea?

Where's the Greg I knew...

--Heig

[Rob Middleton]

I'd still reckon contents of the home directory itself rather than the fact it is root.

Perhaps:

mv /Users/workinguser/* /var/root/.

chown -R root:wheel /var/root

If that works it is just some piece of content inside /var/root

Though perhaps it actually wants the login keychain unlocked. That would be a real pain.

(I'm sure it would be easier if I went and found a 10.8 machine :-) Must be one in the building somewhere by now)

And I misread Heig's last email - that just replacing com.apple.security was not enough. Looks like it wants to see a real login keychain before continuing on to the main System keychain where it should be looking. I expect getting some good contents in /var/root will workaround more than just this specific 10.8 problem .. I'll let someone else find all the bugs, 10.7 works well on my machine!

Rob.

[Heig Gregorian]

No good on that one either...

I suppose I'll still open a bug noting the differences between a "clean" install vs. an upgrade.

--Heig

[Rob Middleton]

Before bug reporting that an upgrade might work do double check it did work.

Note:

$ sudo bash

Password:

bash-3.2# whoami

root

bash-3.2# echo $HOME

/Users/rob

You can be root without the home directory being changed. So check that you did have /var/root as HOME when you saw the upgraded 10.8 work.

I know you saw this earlier, but worth double checking before you bug report.

Rob.

[Heig Gregorian]

Yes, I've doubly checked...

And btw, not that it matters, because we've tested this in the python environment:

$ export HOME=/Users/hgregorian

$ echo $HOME

/Users/hgregorian

...makes everything happy again.

--Heig

[Greg Neagle]

On Jul 26, 2012, at 8:26 PM, Heig Gregorian wrote:

Here's some more fun...I wrote a haiku:

You're so smart, wise guy...

Do you have an idea?

Where's the Greg I knew...

Having success recruiting helpers! And glad to see Rob Middleton again. I guess we needed a suitable puzzle to figure out.

-Greg

[David Vandenborn]

Just a heads up to other users who run into this problem.

Munki (0.8.2.1475.0) was giving me an error while trying to install Microsoft AutoUpdate for Mac v2.3.5 on clients that run 10.8.
The error was 'Certificate used to sign package is not trusted. Use -allowUntrusted to override.'.

Checking the certificate (pkgutil --check-signature) on 10.8 as su would give the result: Status: invalid signature
Checking the certificate on 10.7.4 as su would tell me: Status: no signature

However, updating the clients to 10.8.1 resolves the problem and Microsoft AutoUpdater for Mac gets installed by Munki.
Checking the certificate on 10.8.1 as su now tells me: Status: signed by a certificate trusted by Mac OS X, followed by the Certificate Chain.

David Vandenborn

[Greg Neagle]

Thanks, David. This sounds like it is a version of the issue discussed here: https://groups.google.com/d/topic/munki-dev/AyV1ZMn-xis/discussion

The main issue was that when operating as root, some signed packages seemed to be treated as though they had invalid signatures.

I'm glad to hear that 10.8.1 seems to have addressed it.

-Greg