managedsoftwareupdate download error

[Jason Hatman]

I've added a new machine to Munki.  I've followed the exact same steps I've followed on all of my other machines (they all work fine, by the way).  However, this one will not download the correct manifest. Apple updates work fine, but not the regular manifest.  I've triple-checked the settings. I've removed and reinstalled Munki.  Here's the error.  

************************************************************************************

hq1103128:~ jhatm0$ sudo /usr/local/munki/managedsoftwareupdate -v
Managed Software Update Tool
Copyright 2010-2014 The Munki Project
http://code.google.com/p/munki


Starting...
Checking for available updates...
Getting manifest client_manifest...
Retrieving list of software for this machine...
Download error: http://munkiserver1.orchard.fruit.com/munki_repo/manifests/stable. Failed (22) with:
Headers: {'http_result_code': '000', 'http_result_description': '', 'curl': '(22) The requested URL returned error: 401 Authorization Required'}
ERROR: Could not retrieve manifest client_manifest from the server: Error 22:
ERROR: Could not retrieve managed install primary manifest.
Checking Apple Software Update catalog...
Caching CatalogURL http://10.2.33.127:8088/index.sucatalog
WARNING: Web server did not return an etag. Munki cannot safely resume downloads without etag support on the web server. URL: http://10.2.33.127:8088/index.sucatalog
Skipping Apple Software Update check because sucatalog is unchanged, installed Apple packages are unchanged and we recently did a full check.
ERROR: Could not read plist: /Library/Managed Installs/manifests/client_manifest.plist
ERROR: plist does not exist.
The following Apple Software Updates are available to install:
+ OS X Update-10.9.2
*Restart required
+ Critical Security Update-1.0


Run managedsoftwareupdate --installonly to install the downloaded updates.
Finishing...
Done.

************************************************************************************

I can't understand why it's trying to download from that particular URL.  I can access the correct manifest through a browser just fine, and it asks me to login like it should.  I have basic authentication turned on.  I do have the AdditionalHttpHeaders section in the plist file.  I've pasted a defaults readout of the ManagedInstalls.plist below. This computer, like all others, uses it's own manifest based off of its serial number.

************************************************************************************

hq1103128:~ jhatm0$ defaults read /Library/Preferences/ManagedInstalls
{
AdditionalHttpHeaders = (
"Authorization: Basic XXXXXXXXXXXXXXXXX"
);
AppleSoftwareUpdatesOnly = 0;
ClientIdentifier = W805210DAGY;
DaysBetweenNotifications = 1;
InstallAppleSoftwareUpdates = 1;
InstalledApplePackagesChecksum = 409a69befd238bbd40c93d1ad9f31b48b92d758730d238a6b1ab68d79107252d;
LastAppleSoftwareUpdateCheck = "2014-03-31 14:44:06 +0000";
LastCheckDate = "2014-03-31 16:05:46 +0000";
LastCheckResult = 0;
LastNotifiedDate = "2014-03-31 13:11:43 +0000";
LogFile = "/Library/Managed Installs/Logs/ManagedSoftwareUpdate.log";
LoggingLevel = 1;
ManagedInstallDir = "/Library/Managed Installs";
SoftwareRepoURL = "http://munkiserver1.orchard.fruit.com/munki_repo";
SoftwareUpdateServerURL = "";
SuppressAutoInstall = 0;
SuppressUserNotification = 0;
UseClientCertificate = 0;
}

************************************************************************************

[Gregory Neagle]

Have you experimented with Simian on this machine in the past?

If you have (and even if you haven't), check the contents of /var/root/Library/Preferences/ManagedInstalls.plist...

Settings here or those set via MCX or configuration profiles will override settings in /Library/Preferences/ManagedInstalls.plist.

-Greg

--
You received this message because you are subscribed to the Google Groups "munki-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to munki-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jason Hatman]

There doesn't appear to be a ManagedInstalls.plist file in that directory.

sh-3.2# ls -alF /var/root/Library/Preferences/
total 120
drwx------+ 20 root wheel 680 Mar 31 08:00 ./
drwx------+ 8 root wheel 272 Dec 16 13:45 ../
-rw-------+ 1 root wheel 70 Dec 5 10:16 .GlobalPreferences.plist
drwx------+ 6 root wheel 204 Mar 25 16:28 ByHost/
-rw-------+ 1 root wheel 72 Mar 25 16:30 com.apple.AOSNotification.plist
-rw-------+ 1 root wheel 778 Mar 21 16:21 com.apple.AppleMultitouchTrackpad.plist
-rw-------+ 1 root wheel 104 Dec 5 10:27 com.apple.CoreGraphics.plist
-rw-------+ 1 root wheel 83 Dec 5 10:30 com.apple.InputMethodKit.UserDictionary.plist
-rw-------+ 1 root wheel 141 Dec 16 13:28 com.apple.LaunchServices.plist
drwx------+ 4 root wheel 136 Dec 5 10:30 com.apple.VoiceOver4/
-rw-------+ 1 root wheel 7636 Dec 5 10:30 com.apple.VoiceOver4.local.plist
-rw-------+ 1 root wheel 345 Dec 5 10:27 com.apple.driver.AppleBluetoothMultitouch.mouse.plist
-rw-------+ 1 root wheel 778 Mar 21 16:21 com.apple.driver.AppleBluetoothMultitouch.trackpad.plist
-rw-------+ 1 root wheel 199 Dec 5 10:27 com.apple.driver.AppleHIDMouse.plist
-rw-------+ 1 root staff 1083 Mar 26 12:34 com.apple.recentitems.plist
-rw-r--r--+ 1 root wheel 257 Dec 5 10:21 com.apple.stackshot.plist
-rw-------+ 1 root wheel 141 Mar 31 07:59 com.apple.telemetry.battery.charge-cycle.plist
-rw-------+ 1 root wheel 109 Feb 26 10:43 com.apple.universalaccess.plist
-rw-------+ 1 root wheel 886 Mar 31 08:00 com.apple.xpc.activity.plist
-rw-------+ 1 root wheel 149 Dec 16 09:18 com.microsoft.autoupdate2.plist

[Jason Hatman]

I just configured another machine just like it and it worked fine.  I would love to know what's wrong with this one because reimaging the machine (currently a reasonable option) will not always be an option.  For times sake, however, i may go ahead and nuke it.

[Gregory Neagle]

If you changed the preferences by directly editing /Library/Preferences/ManagedInstalls.plist you may have been running afoul of cfprefd, which aggressively caches preferences in Mavericks and less aggressively caches them in Mountain Lion.

You should always use `defaults write` (or the equivalent CoreFoundation/Foundation methods) to modify live preferences on Mountain Lion+.  File editing will lead to unexpected results.

-Greg

On Mar 31, 2014, at 11:28 AM, Jason Hatman <jason....@gmail.com> wrote:

I just configured another machine just like it and it worked fine.  I would love to know what's wrong with this one because reimaging the machine (currently a reasonable option) will not always be an option.  For times sake, however, i may go ahead and nuke it.

[Jason Hatman]

I currently manage that file using Puppet and CFPropertyList. It's worked very well so far.

[Gregory Neagle]

On Mar 31, 2014, at 12:04 PM, Jason Hatman <jason....@gmail.com> wrote:

I currently manage that file using Puppet and CFPropertyList. It's worked very well so far.

That's probably the source of your issue then.  Puppet is doing direct file modification. 

[Jason Hatman]

It seems to work on all other Mavericks systems.  I was using a Puppet provider that that used the Defaults command, but it didn't play nice with Arrays, which I need for the AdditionalHttpHeaders value.  That's when I moved to CFPropertyList.  I'll experiment some more and see. I may end up having to switch providers again.

[Graham Gilbert]

You're going to have much better luck using profiles with puppet for this sort of thing.

https://github.com/ryanycoleman/rcoleman-mac_profiles_handler

It might not totally suit your needs, but I have a module that will set up Munki for you (amongst other things). At the very least it should give you a place to start from. And now back on topic, as this isn't Munki related.

https://github.com/grahamgilbert/puppet-mac_admin

Graham Gilbert

Sent from my iPhone

On 31 Mar 2014, at 20:13, Jason Hatman <jason....@gmail.com> wrote:

It seems to work on all other Mavericks systems.  I was using a Puppet provider that that used the Defaults command, but it didn't play nice with Arrays, which I need for the AdditionalHttpHeaders value.  That's when I moved to CFPropertyList.  I'll experiment some more and see. I may end up having to switch providers again.

--

[Jason Hatman]

I ended up reimaging the Mavericks machine that was not working.  It worked fine after that.  Today I have another machine doing the same thing, except it's running Mountain Lion.  Same messages as above and same settings, except for the serial number.  I tried manually removing Munki and reinstalling Munki.  After the reinstall, I used the defaults command to create the ManagedInstalls.plist file and I still get the same error.  Are there any other files that I should check?  Where is it getting that URL?

Managed Software Update Tool

Copyright 2010-2014 The Munki Project

http://code.google.com/p/munki

Starting...

Checking for available updates...

    Getting manifest client_manifest...

    Retrieving list of software for this machine...

    Download error: http://munkiserver1.orchard.fruit.com/munki_repo/manifests/stable. Failed (22) with: The requested URL returned error: 401

    Headers: {'http_result_code': '401', 'http_result_description': ''}

ERROR: Could not retrieve manifest client_manifest from the server: Error 22: The requested URL returned error: 401

ERROR: Could not retrieve managed install primary manifest.

Checking Apple Software Update catalog...

    Caching CatalogURL http://10.2.33.127:8088/index.sucatalog

    WARNING: Web server did not return an etag. Munki cannot safely resume downloads without etag support on the web server. URL: http://10.2.33.127:8088/index.sucatalog

    Skipping Apple Software Update check because sucatalog is unchanged, installed Apple packages are unchanged and we recently did a full check.

ERROR: Could not read plist: /Library/Managed Installs/manifests/client_manifest.plist

ERROR: plist does not exist.

    The following Apple Software Updates are available to install:

        + iTunes-11.1.3

        + Java for OS X 2013-005-1.0

        + OS X Update Combined-10.8.5

           *Restart required

        + iTunes-11.1.4

        + iTunes-11.1.5

[Gregory Neagle]

On Apr 9, 2014, at 6:11 AM, Jason Hatman <jason....@gmail.com> wrote:

I ended up reimaging the Mavericks machine that was not working.  It worked fine after that.  Today I have another machine doing the same thing, except it's running Mountain Lion.  Same messages as above and same settings, except for the serial number.  I tried manually removing Munki and reinstalling Munki.  After the reinstall, I used the defaults command to create the ManagedInstalls.plist file and I still get the same error.  Are there any other files that I should check?  Where is it getting that URL?

That URL constructed based on settings in ManagedInstalls.plist, which can exist in three possible places:

Via MCX or profiles, which would leave a cache at /Library/Managed Preferences/ManagedInstalls.plist

/var/root/Library/Preferences/ManagedInstalls.plist

/Library/Preferences/ManagedInstalls.plist

-Greg

[Jason Hatman]

I looked in all three places.  The only place a ManagedInstalls.plist file exists is in /Library/Preferences/.  If I look at the plist file, nowhere in there does it have http://munkiserver1.orchard.fruit.com/munki_repo/manifests/stable.  that URL doesn't even exist. There is no manifest called "stable".  It should be pulling the manifest that is named "QP838122ZE7" because that's what I'm telling it to do in the plist file.  See below.

HQ1088352:~ admin$ defaults read /Library/Preferences/ManagedInstalls

{

    AdditionalHttpHeaders =     (

        "Authorization: Basic XXXXXXXXXXXXXXXXXXXX"

    );

    AppleSoftwareUpdatesOnly = 0;

    ClientIdentifier = QP838122ZE7;

    DaysBetweenNotifications = 1;

    InstallAppleSoftwareUpdates = 1;

    InstalledApplePackagesChecksum = 65ed7f2b2afaef516bddfaed7343ef6e34509caa63aa5a8620dc2b34d480f96b;

    LastAppleSoftwareUpdateCheck = "2014-04-09 14:17:35 +0000";

    LastCheckDate = "2014-04-09 14:17:11 +0000";

    LastCheckResult = 0;

    LastNotifiedDate = "2014-04-08 22:57:15 +0000";

[Jason Hatman]

And that manifest does exist on the server.

munkiserver1:updates admin$ ls -l /Users/Shared/munki_repo/manifests/

total 224

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:04 Adobe_CS5.5_DP

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:07 Adobe_CS5.5_DS

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:07 Adobe_CS5.5_WP

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:07 Adobe_CS5_DS

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:08 Adobe_CS6_DS

-rwxrwxrwx  1 _www  _www   779 Mar  4 16:06 Adobe_CS6_PP

-rw-r--r--  1 _www  _www   779 Mar  7 14:53 Adobe_RemoveAll

-rw-r--r--  1 _www  _www   574 Apr  4 16:08 C07K70WRDY3H

-rw-r--r--  1 _www  _www   883 Mar 20 14:19 OS_X_10.9

-rw-r--r--  1 _www  _www   574 Mar 20 20:42 QP0221DN5PE

-rw-r--r--  1 _www  _www   574 Apr  4 16:57 QP83810SZE7

-rw-r--r--  1 _www  _www   574 Apr  8 16:46 QP838122ZE7

-rw-r--r--  1 _www  _www   574 Apr  8 15:55 QP8390M7ZE7

-rwxrwxrwx  1 _www  _www   611 Mar  5 07:53 RM8332KA0P1

-rw-r--r--  1 _www  _www   578 Mar 21 08:47 VM9oxvzAkq5Y

-rwxrwxrwx  1 _www  _www   611 Mar 10 09:07 VMWVk03uefwgOo7RuvgIlfnrg

-rw-r--r--  1 _www  _www   609 Mar 31 13:37 VMWVk1WmkKk9IJuqqCQ82NuYg

-rw-r--r--  1 _www  _www   610 Mar 31 16:31 VMWVk20GY5tdruuHshoSyVbvQ

-rwxr-xr-x  1 _www  _www   609 Mar 27 11:52 VMWVk3Gjn9lKicnKNtHQkYuYw

-rw-r--r--  1 _www  _www   578 Mar 31 13:09 VMjHbK0SMqGT

-rw-r--r--  1 _www  _www   610 Apr  4 08:18 VMwoqWG7K9Fm

-rw-r--r--  1 _www  _www   574 Mar 31 16:40 W805210DAGY

-rw-r--r--  1 _www  _www   574 Apr  8 15:23 W8942FAQ5PE

-rw-r--r--  1 _www  _www   908 Mar 10 08:54 flip4mac_install

-rwxrwxrwx  1 _www  _www  1226 Apr  8 15:23 fotl_common

-rw-r--r--  1 _www  _www   693 Mar 10 09:29 java7_install

-rwxrwxrwx  1 _www  _www   513 Feb 16 15:57 munki_server

-rwxrwxrwx  1 _www  _www  1033 Feb 28 13:20 symantec_install

[Jason Hatman]

Hold on.  I just found a "ManagedInstalls.A4AA33A9-D74B-5D38-97C6-9679D2E4DB14.plist" file in /var/root/Library/Preferences/ByHost/.   And yes, it was indeed related to a Simian test i ran last year.  I just happened to think to look in the ByHost directory.  I deleted that file and all works as expected now.  Thanks all!

[Gregory Neagle]

Simian!!!!!!!

(Seriously, it's been the cause of lots of issues -- people have no idea what it does to a system and how to reverse it.)

-Greg

On Apr 9, 2014, at 7:59 AM, Jason Hatman <jason....@gmail.com> wrote:

Hold on.  I just found a "ManagedInstalls.A4AA33A9-D74B-5D38-97C6-9679D2E4DB14.plist" file in /var/root/Library/Preferences/ByHost/.   And yes, it was indeed related to a Simian test i ran last year.  I just happened to think to look in the ByHost directory.  I deleted that file and all works as expected now.  Thanks all!

[David Iwanicki]

Greg,

Should the various ByHost(s) should be added to the documentation as a fourth possible place? What priority would it have in regards to the other three?

On Wednesday, April 9, 2014 10:59:41 AM UTC-4, Jason Hatman wrote:

Hold on.  I just found a "ManagedInstalls.A4AA33A9-D74B-5D38-97C6-9679D2E4DB14.plist" file in /var/root/Library/Preferences/ByHost/.   And yes, it was indeed related to a Simian test i ran last year.  I just happened to think to look in the ByHost directory.  I deleted that file and all works as expected now.  Thanks all!

[Gregory Neagle]

This is the same as all OS X preferences.

I did not think to mention ByHost because I was unaware Simian wrote preferences there.

But for _all_ preferences this is the search path in most cases:

MCX/Profiles/Managed Preferences

~/Library/Preferences/ByHost/IDENTIFIER.plist

~/Library/Preferences/IDENTIFIER.plist

~/Library/Preferences/.GlobalPreferences.plist

/Library/Preferences/IDENTIFIER.plist

/Library/Preferences/.GlobalPreferences.plist

(At one time there was also /Network/Library/Preferences -- not sure if this is still checked, and in most cases doesn't exist)

There are actually a few other places: the NSRegistrationDomain and the NSArgumentDomain, but these can usually be ignored.

See:

https://developer.apple.com/library/mac/documentation/Cocoa/Reference/Foundation/Classes/nsuserdefaults_Class/Reference/Reference.html

and

https://developer.apple.com/library/mac/documentation/CoreFoundation/Reference/CFPreferencesUtils/Reference/reference.html

If you really want the in-depth answers.

-Greg

On Apr 9, 2014, at 8:47 AM, David Iwanicki <david.i...@gmail.com> wrote:

Should the various ByHost(s) should be added to this list as a fourth possible place? What priority would it have in regards to the other three?