For Simian, we've just implemented a feature where
managedsoftwareupdate crashes are reported the server with full stack
traces, using supervisor and the report_broken_client script.
The problem I'm seeing is that graceful exits in preflight are causing
managedsoftwareupdate to exit with the same code as any other crash,
such as uncaught exceptions in Munki source. By "graceful exits" I
mean cases where we don't want Munki to execute (e.g. while connected
via a WWAN card), so preflight exits with a non-zero,
managedsoftwareupdate exits with the same code as it does elsewhere,
and supervisor thinks it crashed.
I propose we:
1) define and document exit codes as constants near the top of
managedsoftwareupdate.
and
2) if preflight/postflight exit with non-zero, exit
managedsoftwareupdate with the same code (currently it's always -1,
which OS X sees as 255).
This way, supervisor can easily decipher between a crash (exit code ==
1), a graceful exit, or some other documented exit case, and act
accordingly.
BTW, we plan to submit our supervisor changes to Munki directly, as
this feature could be used with custom report_broken_client scripts in
non-Simian deployments.
On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote: > I propose we: > 1) define and document exit codes as constants near the top of > managedsoftwareupdate. > and > 2) if preflight/postflight exit with non-zero, exit > managedsoftwareupdate with the same code (currently it's always -1, > which OS X sees as 255).
Sounds very reasonable. I'd suggest defining:
A) Currently known errors (no network, avoidable network like airplane wifi, etc.). B) Catch all unknown values (fatal, temporary, etc.). C) Reserved ranges that can be defined in the future, and are guaranteed to otherwise not be used until then.
A and B would probably mean "ranges" of goodness and badness happening, much like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from the most significant digit, and often more detail from the less significant ones, if you care.
That way whatever interprets these values has the best chance of being able to.
On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
>> I propose we:
>> 1) define and document exit codes as constants near the top of
>> managedsoftwareupdate.
>> and
>> 2) if preflight/postflight exit with non-zero, exit
>> managedsoftwareupdate with the same code (currently it's always -1,
>> which OS X sees as 255).
> Sounds very reasonable. I'd suggest defining:
> A) Currently known errors (no network, avoidable network like airplane wifi,
> etc.).
> B) Catch all unknown values (fatal, temporary, etc.).
> C) Reserved ranges that can be defined in the future, and are guaranteed to
> otherwise not be used until then.
> A and B would probably mean "ranges" of goodness and badness happening, much
> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
> the most significant digit, and often more detail from the less significant
> ones, if you care.
> That way whatever interprets these values has the best chance of being able
> to.
So, you can see when Munki crashes (Python exit == 1)
report_broken_clent POSTs stderr (the Python traceback) to our Simian
server. So far we've found 4-5 intermittent/minor bugs in Munki from
such tracebacks, so it's already proving useful.
If widely desired, we could set Munki's default launchd plists
similarly, add to the report_broken_client documentation at
http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
supervisor only attempt to launch report_broken_client if it exists.
Thoughts?
> Internally, our Simian instance is running an enhancement supervisor
> which detects Python crashes and uploads the traceback to our server.
> This has already exposed a couple of bugs, one of which was fixed
> here: https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
> I'll submit the supervisor changes to Munki soon...
> - Justin
> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
>>> I propose we:
>>> 1) define and document exit codes as constants near the top of
>>> managedsoftwareupdate.
>>> and
>>> 2) if preflight/postflight exit with non-zero, exit
>>> managedsoftwareupdate with the same code (currently it's always -1,
>>> which OS X sees as 255).
>> Sounds very reasonable. I'd suggest defining:
>> A) Currently known errors (no network, avoidable network like airplane wifi,
>> etc.).
>> B) Catch all unknown values (fatal, temporary, etc.).
>> C) Reserved ranges that can be defined in the future, and are guaranteed to
>> otherwise not be used until then.
>> A and B would probably mean "ranges" of goodness and badness happening, much
>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
>> the most significant digit, and often more detail from the less significant
>> ones, if you care.
>> That way whatever interprets these values has the best chance of being able
>> to.
That seems worthwhile to have as standard to me. I should be capturing such info to a central server .. I'm not yet.
Do we need to make a change to the launchd scripts? Or can we make --error-exec as specified the default unless overridden?
I'm undecided whether it is better to:
- have the launchd plist arguments quite explicit
- not change the launchd plists unless we really need to, as that increments the subpackage version & requires a restart on upgrades.
Rob.
On 13/11/2012, at 6:47 AM, Justin McWilliams wrote:
> So, you can see when Munki crashes (Python exit == 1)
> report_broken_clent POSTs stderr (the Python traceback) to our Simian
> server. So far we've found 4-5 intermittent/minor bugs in Munki from
> such tracebacks, so it's already proving useful.
> If widely desired, we could set Munki's default launchd plists
> similarly, add to the report_broken_client documentation at
> http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
> supervisor only attempt to launch report_broken_client if it exists.
> Thoughts?
>> Internally, our Simian instance is running an enhancement supervisor
>> which detects Python crashes and uploads the traceback to our server.
>> This has already exposed a couple of bugs, one of which was fixed
>> here: https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
>> I'll submit the supervisor changes to Munki soon...
>> - Justin
>> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
>>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
>>>> I propose we:
>>>> 1) define and document exit codes as constants near the top of
>>>> managedsoftwareupdate.
>>>> and
>>>> 2) if preflight/postflight exit with non-zero, exit
>>>> managedsoftwareupdate with the same code (currently it's always -1,
>>>> which OS X sees as 255).
>>> Sounds very reasonable. I'd suggest defining:
>>> A) Currently known errors (no network, avoidable network like airplane wifi,
>>> etc.).
>>> B) Catch all unknown values (fatal, temporary, etc.).
>>> C) Reserved ranges that can be defined in the future, and are guaranteed to
>>> otherwise not be used until then.
>>> A and B would probably mean "ranges" of goodness and badness happening, much
>>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
>>> the most significant digit, and often more detail from the less significant
>>> ones, if you care.
>>> That way whatever interprets these values has the best chance of being able
>>> to.
I second having more information regarding a run and how it went. I'd vote
for not updating the launchd items unless there is a solid reason to do so.
If it is the best way of doing it, then update the launchd item.
Nate
On Mon, Nov 12, 2012 at 3:57 PM, Rob Middleton <rrmiddle...@gmail.com>wrote:
> That seems worthwhile to have as standard to me. I should be capturing
> such info to a central server .. I'm not yet.
> Do we need to make a change to the launchd scripts? Or can we make
> --error-exec as specified the default unless overridden?
> I'm undecided whether it is better to:
> - have the launchd plist arguments quite explicit
> - not change the launchd plists unless we really need to, as that
> increments the subpackage version & requires a restart on upgrades.
> Rob.
> On 13/11/2012, at 6:47 AM, Justin McWilliams wrote:
> > So, you can see when Munki crashes (Python exit == 1)
> > report_broken_clent POSTs stderr (the Python traceback) to our Simian
> > server. So far we've found 4-5 intermittent/minor bugs in Munki from
> > such tracebacks, so it's already proving useful.
> > If widely desired, we could set Munki's default launchd plists
> > similarly, add to the report_broken_client documentation at
> > http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
> > supervisor only attempt to launch report_broken_client if it exists.
> > Thoughts?
> > - Justin
> > On Wed, Oct 24, 2012 at 12:24 PM, Justin McWilliams <o...@google.com>
> wrote:
> >> FYI, this is done here:
> >> Internally, our Simian instance is running an enhancement supervisor
> >> which detects Python crashes and uploads the traceback to our server.
> >> This has already exposed a couple of bugs, one of which was fixed
> >> here:
> https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
> >> I'll submit the supervisor changes to Munki soon...
> >> - Justin
> >> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <
> alieu...@google.com> wrote:
> >>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams
> wrote:
> >>>> I propose we:
> >>>> 1) define and document exit codes as constants near the top of
> >>>> managedsoftwareupdate.
> >>>> and
> >>>> 2) if preflight/postflight exit with non-zero, exit
> >>>> managedsoftwareupdate with the same code (currently it's always -1,
> >>>> which OS X sees as 255).
> >>> Sounds very reasonable. I'd suggest defining:
> >>> A) Currently known errors (no network, avoidable network like airplane
> wifi,
> >>> etc.).
> >>> B) Catch all unknown values (fatal, temporary, etc.).
> >>> C) Reserved ranges that can be defined in the future, and are
> guaranteed to
> >>> otherwise not be used until then.
> >>> A and B would probably mean "ranges" of goodness and badness
> happening, much
> >>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information
> from
> >>> the most significant digit, and often more detail from the less
> significant
> >>> ones, if you care.
> >>> That way whatever interprets these values has the best chance of being
> able
> >>> to.
> I second having more information regarding a run and how it went. I'd vote for not updating the launchd items unless there is a solid reason to do so. If it is the best way of doing it, then update the launchd item.
> Nate
> On Mon, Nov 12, 2012 at 3:57 PM, Rob Middleton <rrmiddle...@gmail.com> wrote:
> That seems worthwhile to have as standard to me. I should be capturing such info to a central server .. I'm not yet.
> Do we need to make a change to the launchd scripts? Or can we make --error-exec as specified the default unless overridden?
> I'm undecided whether it is better to:
> - have the launchd plist arguments quite explicit
> - not change the launchd plists unless we really need to, as that increments the subpackage version & requires a restart on upgrades.
> Rob.
> On 13/11/2012, at 6:47 AM, Justin McWilliams wrote:
> > So, you can see when Munki crashes (Python exit == 1)
> > report_broken_clent POSTs stderr (the Python traceback) to our Simian
> > server. So far we've found 4-5 intermittent/minor bugs in Munki from
> > such tracebacks, so it's already proving useful.
> > If widely desired, we could set Munki's default launchd plists
> > similarly, add to the report_broken_client documentation at
> > http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
> > supervisor only attempt to launch report_broken_client if it exists.
> > Thoughts?
> >> Internally, our Simian instance is running an enhancement supervisor
> >> which detects Python crashes and uploads the traceback to our server.
> >> This has already exposed a couple of bugs, one of which was fixed
> >> here: https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
> >> I'll submit the supervisor changes to Munki soon...
> >> - Justin
> >> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
> >>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
> >>>> I propose we:
> >>>> 1) define and document exit codes as constants near the top of
> >>>> managedsoftwareupdate.
> >>>> and
> >>>> 2) if preflight/postflight exit with non-zero, exit
> >>>> managedsoftwareupdate with the same code (currently it's always -1,
> >>>> which OS X sees as 255).
> >>> Sounds very reasonable. I'd suggest defining:
> >>> A) Currently known errors (no network, avoidable network like airplane wifi,
> >>> etc.).
> >>> B) Catch all unknown values (fatal, temporary, etc.).
> >>> C) Reserved ranges that can be defined in the future, and are guaranteed to
> >>> otherwise not be used until then.
> >>> A and B would probably mean "ranges" of goodness and badness happening, much
> >>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
> >>> the most significant digit, and often more detail from the less significant
> >>> ones, if you care.
> >>> That way whatever interprets these values has the best chance of being able
> >>> to.
Now: make sure you are doing the right thing for non-boot volumes, and test the behaviors on all supported OS versions (10.5-10.8). And realize that if Munki itself is doing the install, unloading the wrong launchdaemon will kill the managedsoftwareupdate process (as it was launched via a launchdaemon).
I won't go so far as to say dynamically updating all the launchagents and launchdaemons is impossible, but it's certainly very hard to do completely correctly. A restart is safer and is much more likely to be successful.
> When modifying LaunchAgents a reboot is generally needed, I guess.
> And after writing this I noticed that there actually is a launchagent which may need the same mod:
> So, can we update all launchd-items AND have those changes be effective immediately without rebooting?
No. You cannot reliably update launchagents for all users without all users logging out. Since the installer can't require that, a restart is what must be done.
> On Nov 12, 2012, at 11:09 PM, Nate <nate.wa...@gmail.com> wrote:
>> I second having more information regarding a run and how it went. I'd vote for not updating the launchd items unless there is a solid reason to do so. If it is the best way of doing it, then update the launchd item.
>> Nate
>> On Mon, Nov 12, 2012 at 3:57 PM, Rob Middleton <rrmiddle...@gmail.com> wrote:
>> That seems worthwhile to have as standard to me. I should be capturing such info to a central server .. I'm not yet.
>> Do we need to make a change to the launchd scripts? Or can we make --error-exec as specified the default unless overridden?
>> I'm undecided whether it is better to:
>> - have the launchd plist arguments quite explicit
>> - not change the launchd plists unless we really need to, as that increments the subpackage version & requires a restart on upgrades.
>> Rob.
>> On 13/11/2012, at 6:47 AM, Justin McWilliams wrote:
>> > So, you can see when Munki crashes (Python exit == 1)
>> > report_broken_clent POSTs stderr (the Python traceback) to our Simian
>> > server. So far we've found 4-5 intermittent/minor bugs in Munki from
>> > such tracebacks, so it's already proving useful.
>> > If widely desired, we could set Munki's default launchd plists
>> > similarly, add to the report_broken_client documentation at
>> > http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
>> > supervisor only attempt to launch report_broken_client if it exists.
>> > Thoughts?
>> >> Internally, our Simian instance is running an enhancement supervisor
>> >> which detects Python crashes and uploads the traceback to our server.
>> >> This has already exposed a couple of bugs, one of which was fixed
>> >> here: https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
>> >> I'll submit the supervisor changes to Munki soon...
>> >> - Justin
>> >> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
>> >>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
>> >>>> I propose we:
>> >>>> 1) define and document exit codes as constants near the top of
>> >>>> managedsoftwareupdate.
>> >>>> and
>> >>>> 2) if preflight/postflight exit with non-zero, exit
>> >>>> managedsoftwareupdate with the same code (currently it's always -1,
>> >>>> which OS X sees as 255).
>> >>> Sounds very reasonable. I'd suggest defining:
>> >>> A) Currently known errors (no network, avoidable network like airplane wifi,
>> >>> etc.).
>> >>> B) Catch all unknown values (fatal, temporary, etc.).
>> >>> C) Reserved ranges that can be defined in the future, and are guaranteed to
>> >>> otherwise not be used until then.
>> >>> A and B would probably mean "ranges" of goodness and badness happening, much
>> >>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
>> >>> the most significant digit, and often more detail from the less significant
>> >>> ones, if you care.
>> >>> That way whatever interprets these values has the best chance of being able
>> >>> to.
> Now: make sure you are doing the right thing for non-boot volumes, and test the behaviors on all supported OS versions (10.5-10.8). And realize that if Munki itself is doing the install, unloading the wrong launchdaemon will kill the managedsoftwareupdate process (as it was launched via a launchdaemon).
> I won't go so far as to say dynamically updating all the launchagents and launchdaemons is impossible, but it's certainly very hard to do completely correctly. A restart is safer and is much more likely to be successful.
>> When modifying LaunchAgents a reboot is generally needed, I guess.
>> And after writing this I noticed that there actually is a launchagent which may need the same mod:
>> So, can we update all launchd-items AND have those changes be effective immediately without rebooting?
> No. You cannot reliably update launchagents for all users without all users logging out. Since the installer can't require that, a restart is what must be done.
>> -MiqViq
>> On Nov 12, 2012, at 11:09 PM, Nate <nate.wa...@gmail.com> wrote:
>>> I second having more information regarding a run and how it went. I'd vote for not updating the launchd items unless there is a solid reason to do so. If it is the best way of doing it, then update the launchd item.
>>> Nate
>>> On Mon, Nov 12, 2012 at 3:57 PM, Rob Middleton <rrmiddle...@gmail.com> wrote:
>>> That seems worthwhile to have as standard to me. I should be capturing such info to a central server .. I'm not yet.
>>> Do we need to make a change to the launchd scripts? Or can we make --error-exec as specified the default unless overridden?
>>> I'm undecided whether it is better to:
>>> - have the launchd plist arguments quite explicit
>>> - not change the launchd plists unless we really need to, as that increments the subpackage version & requires a restart on upgrades.
>>> Rob.
>>> On 13/11/2012, at 6:47 AM, Justin McWilliams wrote:
>>> > So, you can see when Munki crashes (Python exit == 1)
>>> > report_broken_clent POSTs stderr (the Python traceback) to our Simian
>>> > server. So far we've found 4-5 intermittent/minor bugs in Munki from
>>> > such tracebacks, so it's already proving useful.
>>> > If widely desired, we could set Munki's default launchd plists
>>> > similarly, add to the report_broken_client documentation at
>>> > http://code.google.com/p/munki/wiki/ReportBrokenClient , and have
>>> > supervisor only attempt to launch report_broken_client if it exists.
>>> > Thoughts?
>>> >> Internally, our Simian instance is running an enhancement supervisor
>>> >> which detects Python crashes and uploads the traceback to our server.
>>> >> This has already exposed a couple of bugs, one of which was fixed
>>> >> here: https://code.google.com/p/munki/source/detail?r=cc4ddcfb4df358ae7426f...
>>> >> I'll submit the supervisor changes to Munki soon...
>>> >> - Justin
>>> >> On Thu, Oct 18, 2012 at 11:25 AM, Anthony Lieuallen <alieu...@google.com> wrote:
>>> >>> On Wednesday, October 17, 2012 5:48:02 PM UTC-4, Justin McWilliams wrote:
>>> >>>> I propose we:
>>> >>>> 1) define and document exit codes as constants near the top of
>>> >>>> managedsoftwareupdate.
>>> >>>> and
>>> >>>> 2) if preflight/postflight exit with non-zero, exit
>>> >>>> managedsoftwareupdate with the same code (currently it's always -1,
>>> >>>> which OS X sees as 255).
>>> >>> Sounds very reasonable. I'd suggest defining:
>>> >>> A) Currently known errors (no network, avoidable network like airplane wifi,
>>> >>> etc.).
>>> >>> B) Catch all unknown values (fatal, temporary, etc.).
>>> >>> C) Reserved ranges that can be defined in the future, and are guaranteed to
>>> >>> otherwise not be used until then.
>>> >>> A and B would probably mean "ranges" of goodness and badness happening, much
>>> >>> like HTTP has 2xx, 3xx, 4xx; you can get a high level of information from
>>> >>> the most significant digit, and often more detail from the less significant
>>> >>> ones, if you care.
>>> >>> That way whatever interprets these values has the best chance of being able
>>> >>> to.
Python headers are required to build Mercurial
Complete output from command python setup.py egg_info:
running egg_info
writing pip-egg-info/mercurial.egg-info/PKG-INFO
writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
Python headers are required to build Mercurial
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /Users/Shared/munkiwebadmin_env/build/mercurial
Storing complete log in /Users/macadmin/.pip/pip.log
Here is the Log
------------------------------------------------------------
/Users/Shared/munkiwebadmin_env/bin/pip run on Wed Nov 14 10:18:54 2012
Downloading/unpacking mercurial
Running setup.py egg_info for package mercurial
running egg_info
writing pip-egg-info/mercurial.egg-info/PKG-INFO
writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
Python headers are required to build Mercurial
Complete output from command python setup.py egg_info:
running egg_info
writing pip-egg-info/mercurial.egg-info/PKG-INFO
writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
warning: manifest_maker: standard file '-c' not found
Python headers are required to build Mercurial
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /Users/Shared/munkiwebadmin_env/build/mercurial
Exception information:
Traceback (most recent call last):
File "/Users/Shared/munkiwebadmin_env/lib/python2.7/site-packages/pip-1.2.1-py2. 7.egg/pip/basecommand.py", line 107, i$
status = self.run(options, args)
File "/Users/Shared/munkiwebadmin_env/lib/python2.7/site-packages/pip-1.2.1-py2. 7.egg/pip/commands/install.py", line 2$
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/Users/Shared/munkiwebadmin_env/lib/python2.7/site-packages/pip-1.2.1-py2. 7.egg/pip/req.py", line 1042, in prepa$
req_to_install.run_egg_info()
File "/Users/Shared/munkiwebadmin_env/lib/python2.7/site-packages/pip-1.2.1-py2. 7.egg/pip/req.py", line 236, in run_eg$
command_desc='python setup.py egg_info')
File "/Users/Shared/munkiwebadmin_env/lib/python2.7/site-packages/pip-1.2.1-py2. 7.egg/pip/util.py", line 612, in call_$
% (command_desc, proc.returncode, cwd))
InstallationError: Command python setup.py egg_info failed with error code 1 in /Users/Shared/munkiwebadmin_env/build/me$
> writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
> writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
> warning: manifest_maker: standard file '-c' not found
> Python headers are required to build Mercurial
> ----------------------------------------
> Command python setup.py egg_info failed with error code 1 in /Users/Shared/munkiwebadmin_env/build/mercurial
> Storing complete log in /Users/macadmin/.pip/pip.log
> Here is the Log
> ------------------------------------------------------------
> /Users/Shared/munkiwebadmin_env/bin/pip run on Wed Nov 14 10:18:54 2012
> Downloading/unpacking mercurial
> Running setup.py egg_info for package mercurial
> running egg_info
> writing pip-egg-info/mercurial.egg-info/PKG-INFO
> writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
> writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
> warning: manifest_maker: standard file '-c' not found
> Python headers are required to build Mercurial
> Complete output from command python setup.py egg_info:
>> writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
>> writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
>> warning: manifest_maker: standard file '-c' not found
>> Python headers are required to build Mercurial
>> ----------------------------------------
>> Command python setup.py egg_info failed with error code 1 in /Users/Shared/munkiwebadmin_env/build/mercurial
>> Storing complete log in /Users/macadmin/.pip/pip.log
>> Here is the Log
>> ------------------------------------------------------------
>> /Users/Shared/munkiwebadmin_env/bin/pip run on Wed Nov 14 10:18:54 2012
>> Downloading/unpacking mercurial
>> Running setup.py egg_info for package mercurial
>> running egg_info
>> writing pip-egg-info/mercurial.egg-info/PKG-INFO
>> writing top-level names to pip-egg-info/mercurial.egg-info/top_level.txt
>> writing dependency_links to pip-egg-info/mercurial.egg-info/dependency_links.txt
>> warning: manifest_maker: standard file '-c' not found
>> Python headers are required to build Mercurial
>> Complete output from command python setup.py egg_info:
