roadmap

[Preston Holmes]

I personally came to the conclusion that I would focus my scripting
skills on python about a year ago as the best language for all my
diverse projects, I've been very happy with that decision and continue
to be pleasantly surprised by python at many a turn.

Certainly it seems like there is a core of others who agree when it
comes to Mac admin stuff (InstaDMG, Munki, this project).

Is there any roadmap or todo list for this project?

I think there is the assumption out there that pymacadmin == crankd

The fact that the core module is installed in /lib/appsupport/crankd
instead of site-packages struck me as odd

I'm interested in contributing to the core library - but given that
everyone is tremendously busy I understand that there isn't always
time to write up exhaustive roadmaps etc.

to kick things off a bit, I'm going to list a brainstorm of mac os
technologies that might be good candidates for coverage in a core
library, and hopefully others will add to, suggest outlines of
possible implementation, or suggest features of the below that would
be the most useful to expose. I'd be happy to clean up such a
discussion and turn it into a wiki page.

- hdiutil (Karl's object from InstaDMG seems like a great start)
- ACLs : add, remove, and propagate

- diskutil
ie:
d = DiskUtil()
d.disks['Macintosh HD'].uuid
>>11111111-2222-3333-4444-555555555555
d.disks['Macintosh HD'].eject()
etc

- dscl
- pmset
- working with finder aliases and alis plist data
- launch services (wrapper for lsregister, python version of duti)
- ioreg

- improve plistlib to handle binary plists (via plutil) and get back
into OSX/python trunk

some others that are probably easiest to keep using the command line
syntax?

ASR
installer
launchctl

-Preston

[Nigel Kersten]

I've actually got a lot of this done that I should check in and clean up.

I was attempting to provide a consistent interface for disks and
images, so you can do something like:

import macdisk

print MacDisk.Partitions()
print MacDisk.WholeDiskDeviceIds()
print MacDisk.WholeDisks()
print MacDisk.MountedVolumeNames()
print MacDisk.AttachedImages()
MacDisk.UnmountAllDiskImages(force=False)

source = MacDisk.Image(imagepath="foo.dmg")
target = MacDisk.Disk("disk0s2")

MacDisk.clone(source, target)

I ran into some design issues getting encrypted image support done in
a clean manner, but a lot of it is quite functional.

I was kind of waiting to see if the relevant private frameworks were
made public before pushing ahead, and then forgot about it all.



>
>  - dscl

If only we had an ObjC DirectoryServices API...

>  - pmset
>  - working with finder aliases and alis plist data
>  - launch services (wrapper for lsregister, python version of duti)
>  - ioreg
>
>  - improve plistlib to handle binary plists (via plutil) and get back
> into OSX/python trunk

I don't see why you'd do this when you can just import Foundation and
use all the NSDictionary to/from file methods?

>
> some others that are probably easiest to keep using the command line
> syntax?
>
> ASR
> installer
> launchctl

If only we had an ObjC launchd framework....

>
> -Preston
>
>
> >
>

[Preston Holmes]

On Jul 17, 2009, at 11:44 AM, Nigel Kersten wrote:

>
> On Fri, Jul 17, 2009 at 11:30 AM, Preston Holmes<sanroq...@gmail.com
> > wrote:
>>
>> - diskutil
>> ie:
>> d = DiskUtil()
>> d.disks['Macintosh HD'].uuid
>> >>11111111-2222-3333-4444-555555555555
>> d.disks['Macintosh HD'].eject()
>> etc
>
> I've actually got a lot of this done that I should check in and
> clean up.
>

Yes please ;-)

>
>
>
>>
>> - dscl
>
> If only we had an ObjC DirectoryServices API...

While preferred - seems that some of the more common/useful tasks
could be done by just wrapping dscl

>
>> - pmset
>> - working with finder aliases and alis plist data
>> - launch services (wrapper for lsregister, python version of duti)
>> - ioreg
>>
>> - improve plistlib to handle binary plists (via plutil) and get back
>> into OSX/python trunk
>
> I don't see why you'd do this when you can just import Foundation and
> use all the NSDictionary to/from file methods?

1) because plistlib is out there an used in code
2) because plistlib.readPlist(pathOrFile) is so simple

if getting changes into plistlib isn't feasible, then a pymacadmin
module that replaces its ease of use with calls to Foundation would
seem nearly as useful.

>
>>
>> some others that are probably easiest to keep using the command line
>> syntax?
>>
>> ASR
>> installer
>> launchctl
>
> If only we had an ObjC launchd framework....

again, in the absence of...
given python's good support for subprocesses, I don't see wrapping as
too objectionable

there is also always higher level goodness around monitoring and
reporting launchd events.

-Preston

[SABRE Robotics]

On 17-Jul-09, at 12:44 PM, Nigel Kersten wrote:

...

> If only we had an ObjC DirectoryServices API...

...

We might, actually.

Included in Apple's open-source DSTools project (do a find in page for
"DSTools" at http://www.opensource.apple.com/release/mac-os-x-1057/ )
is a project called "DSObjCWrappers". The current version (112.1)
appears to be undocumented (and I was hoping to figure it out sometime
if I had the time).

Perhaps more interesting than that is that it appears to live on
Leopard machines in /System/Library/PrivateFrameworks/
DSObjCWrappers.framework.

As far as I can tell, they are the same version. (I'm running OS X
10.5.7, and the open source download is the one that corresponds to
it.) /System/Library/PrivateFrameworks/DSObjCWrappers.framework/
Versions/A/Resources/version.plist contains these keys (among others):

<key>CFBundleVersion</key>
<string>1.3</string>
<key>ProjectName</key>
<string>DSTools</string>
<key>SourceVersion</key>
<string>1120100</string>

I suspect that means that source version is 112.1 of the DSTools
project.

So, the source is open, but the framework is private. It may just be
safe to use, though. [Assuming we can get at it through the PyObjC
bridge].

Cheers,
Clinton

[Jeremy Reichman]

I haven't been paying much attention, but can pymacadmin (by this or another
name) be a package that people just contribute modules to? Is that what's
already happening, or not? That way, it can be planned out, or not.

I've toyed with making modules in a package and then symlinking them for use
as command line tools, when that is needed for stuff that must be
standalone. It works. (I did a little bit of work on this across major Mac
OS X versions, since Tiger and Leopard shipped with different Python
versions and site-packages paths. I think I might have successfully use a
.pth file to jump between 2.3 and 2.5.)

Also, I only put stuff in site-packages, and I've taken to the use of a
package to namespace what I've written. The import statement may get
slightly longer but it's not usually a big deal.

Eventually, maybe some/all of this can just be contributed back to Python
itself. I think at that potential future time, if not sooner, issues of
namespacing and whatnot come up.

For that reason (as well as others), I'd suggest we try to follow PEP 8 for
contributions if it is not being done now. I try do that for my own stuff.

There is my random contribution of the month. You're welcome to it. :)

--
Jeremy

[Greg Neagle]

On Jul 17, 2009, at 12:18 PM, Preston Holmes wrote:

>>>
>>> - improve plistlib to handle binary plists (via plutil) and get back
>>> into OSX/python trunk
>>
>> I don't see why you'd do this when you can just import Foundation and
>> use all the NSDictionary to/from file methods?
>
> 1) because plistlib is out there an used in code
> 2) because plistlib.readPlist(pathOrFile) is so simple

It's really trivial to replace plistlb.readPlist() with something that
can handle binary plists as well as XML/text ones:

def readPlist(plistfile):
"""Read a plist, return a dict.
This method can deal with binary plists,
whereas plistlib cannot.

Args:
plistfile: Path to plist file to read

Returns:
dict of plist contents.
"""
from Foundation import NSDictionary
return NSDictionary.dictionaryWithContentsOfFile_(plistfile)

You don't need to modify/replace plistlib.writePlist() because it's
fine to write out a plist in XML/text format.

-Greg

[Preston Holmes]

On Jul 17, 2009, at 2:41 PM, Greg Neagle wrote:

>
>
> On Jul 17, 2009, at 12:18 PM, Preston Holmes wrote:
>
>>>>
>>>> - improve plistlib to handle binary plists (via plutil) and get
>>>> back
>>>> into OSX/python trunk
>>>
>>> I don't see why you'd do this when you can just import Foundation
>>> and
>>> use all the NSDictionary to/from file methods?
>>
>> 1) because plistlib is out there an used in code
>> 2) because plistlib.readPlist(pathOrFile) is so simple
>
> It's really trivial to replace plistlb.readPlist() with something that
> can handle binary plists as well as XML/text ones:

Granted - but I assume (perhaps incorrectly) that a design goal of a
project like pymacadmin would be to present a *pythonic* option for
accessing the technology.

That is, that PyObjC usage/syntax be limited wherever possible to the
implementation of a module, and not required for its usage.

My point wasn't whether the implementation was hard or easy - but just
that the current pythonic interface to plists is currently (half)
broken.

I myself have only dug shallow little pits in the sea of cocoa, and I
look to pymacadmin (as a user) as something that can protect me from
cocoa (and as a potential contributor as something of an inspiration
for digging deeper into cocoa/PyObjC).

-Preston

> -Greg
>
>
> >

[Nigel Kersten]

People should maybe wait and see about my "if only ..." statements ....

I feel like plistlib in all scripting languages is ultimately doomed
due to the binary format issue, and will end up being relegated for
use on non OS X platforms where you need to manipulate plists from
scripting languages.

I don't believe Apple have ever documented the binary plist format have they?

[Karl Kuehn]

On Jul 17, 2009, at 12:18 PM, Preston Holmes wrote:

>> I don't see why you'd do this when you can just import Foundation
>> and use all the NSDictionary to/from file methods?
>
> 1) because plistlib is out there an used in code
> 2) because plistlib.readPlist(pathOrFile) is so simple
>
> if getting changes into plistlib isn't feasible, then a pymacadmin
> module that replaces its ease of use with calls to Foundation would
> seem nearly as useful.

I agree with Nigel that plistlib is not worth using at this point.
And the Foundation replacements are easy drop-ins:

import Foundation
plist = Foundation.NSDictionary.dictionaryWithContentsOfFile_(plistPath)

Or if you are not sure that the root is a NSDictionary (or want more
to know the format):

import Foundation
plistData = Foundation.NSData.dataWithContentsOfFile_(plistPath)
dataObject, plistFormat, error =
Foundation.NSPropertyListSerialization.propertyListFromData_mutabilityOption_format_errorDescription_
(plistData, Foundation.NSPropertyListMutableContainers)

And if you use the later form, then you can write things back out in
any format the Foundation supports:

import Foundation
plistData, error =
Foundation.NSPropertyListSerialization.dataFromPropertyList_format_errorDescription_
(dataObject, Foundation.NSPropertyListBinaryFormat_v1_0)
plistData.writeToFile_atomically_(plistPath, TRUE)

And the difference between wrapping a command line utility and
actually using a binary interface is huge. The latter is much more
dependable.

--
Karl Kuehn
lar...@softhome.net

[Preston Holmes]

On Jul 17, 2009, at 3:22 PM, Karl Kuehn wrote:

> I agree with Nigel that plistlib is not worth using at this point.
> And the Foundation replacements are easy drop-ins:

Despite my reference to plutil, my basic point is that the need for a
working plistlib, or something like it persists.

I don't think there is anyway that

"Foundation
.NSPropertyListSerialization
.propertyListFromData_mutabilityOption_format_errorDescription_"

can be considered pythonic.

plistlib has been dropped from python as of 2.6 (yet I still see it in
[REDACTED])

http://svn.python.org/view/python/tags/r261/Lib/plat-mac/

Apple is in the business of extending python with mac/darwin specific
modules:

http://www.opensource.apple.com/source/python_modules/python_modules-12/

So either a plistlib replacement that uses foundation could be
contributed to Darwin, or to PyMacAdmin - but I'll stick my assertion
that PyObjC is not a drop in replacement for plistlib in general
purpose python scripts.

-Preston

[Nigel Kersten]

On Fri, Jul 17, 2009 at 4:22 PM, Preston Holmes<sanroq...@gmail.com> wrote:
>
>
> On Jul 17, 2009, at 3:22 PM, Karl Kuehn wrote:
>
>>       I agree with Nigel that plistlib is not worth using at this point.
>> And the Foundation replacements are easy drop-ins:
>
> Despite my reference to plutil, my basic point is that the need for a
> working plistlib, or something like it persists.
>
> I don't think there is anyway that
>
> "Foundation
> .NSPropertyListSerialization
> .propertyListFromData_mutabilityOption_format_errorDescription_"
>
> can be considered pythonic.

As soon as you start working with PyObjC your code isn't really pythonic at all.

Have you seen any of bbum's WWDC talks on PyObjC or the mailing list
discussions around this?

It's really hard to come up with a maintainable solution for this stuff.

[Preston Holmes]

On Jul 17, 2009, at 4:33 PM, Nigel Kersten wrote:

> As soon as you start working with PyObjC your code isn't really
> pythonic at all.

I guess I'm really not communicating my point well.

I'm wondering whether anyone else agrees that PyMacAdmin module, as a
python module should generally offer pythonic functions and classes to
interact with Apple system technologies.

That the *implementation* of these functions and classes may heavily
use PyObjC, but that the PyObjC should, whenever possible, not be
exposed in a module's API.

To use our current example of plists

plists are heavily used in mac administration and are not going away

It would be useful for for Mac Python scripters to have a module that
did 4 simple things

convert from a python dictionary to and from a plist in either string
or file format

I think it reasonable for Mac Python scripters to be able to do this
without using PyObjC foundation calls *directly* in their script

I'm thinking of new potential users of Python and PyMacAdmin - not all
of them are going to come in wanting to simultaneously grok both
python and Cocoa frameworks

Sorry to keep banging on this, I'm just trying to get a read on
whether people think the USAGE of PyMacAdmin should be pythonic (I
think so) regardless of whether it uses non-pythonic PyObjC - all I'm
hearing is "Just use Foundation"

-Preston

[Greg Neagle]

Yes to Pythonic wrappers around Mac OS X frameworks for commonly-used
admin tools.

On the specific point of plists, in my opinion there is no need to
replace/augment plistlib for _writing_ plists unless it's going away
in future versions of Python.

For the specific task of reading plists in order to access OS X
defaults, though, what's really needed is a wrapper around
CFPreferences so that the various preference domains are properly
honored. Something like this (which was written to ape the existing
defaults tool, not to be a Pythonic function):

#!/usr/bin/env python
"""
effective_defaults.py

Created by Greg Neagle on 2009-06-26.
"""

import sys
import CoreFoundation

try:
preferenceDomain = sys.argv[1]
keyName = sys.argv[2]
except:
print "Usage: %s <domain> <key>" % sys.argv[0]
print "\tWhere <domain> is a valid preferences (defaults) domain,
and"
print "\twhere <key> is a valid preferences key"
print
print"\tExample: %s com.apple.screensaver askForPassword" %
sys.argv[0]
exit(-1)

print CoreFoundation.CFPreferencesCopyAppValue(keyName,
preferenceDomain)
if CoreFoundation.CFPreferencesAppValueIsForced(keyName,
preferenceDomain):
print "*** %s is managed always by MCX ***" % keyName

[Nigel Kersten]

On Fri, Jul 17, 2009 at 5:42 PM, Preston Holmes<sanroq...@gmail.com> wrote:
>
>
> On Jul 17, 2009, at 4:33 PM, Nigel Kersten wrote:
>
>> As soon as you start working with PyObjC your code isn't really
>> pythonic at all.
>
> I guess I'm really not communicating my point well.

I think you have now though.

I agree completely that modules we all develop should be as pythonic
as possible.

I guess I just feel like reading a plist is trivial enough with
Foundation that I've never really thought about putting a module
together for it, but I do think you're right that a better plistlib
would be really useful for a lot of people.

[Jeremy Reichman]

On 7/17/2009 8:42 PM, "Preston Holmes" <sanroq...@gmail.com> wrote:

> I guess I'm really not communicating my point well.
>
> I'm wondering whether anyone else agrees that PyMacAdmin module, as a
> python module should generally offer pythonic functions and classes to
> interact with Apple system technologies.
>
> That the *implementation* of these functions and classes may heavily
> use PyObjC, but that the PyObjC should, whenever possible, not be
> exposed in a module's API.

I was agreeing with you both before and after you re-explained. I'm pretty
happy to let others delve into PyObjC, myself. Having a Pythonic wrapper
around the system stuff while simultaneously making things work
better/faster/more broadly would be fantastic.

--
Jeremy

[Chris Adams]

On Fri, Jul 17, 2009 at 8:42 PM, Preston Holmes<sanroq...@gmail.com> wrote:
> Sorry to keep banging on this, I'm just trying to get a read on
> whether people think the USAGE of PyMacAdmin should be pythonic (I
> think so) regardless of whether it uses non-pythonic PyObjC - all I'm
> hearing is "Just use Foundation"

I missed out on most of this discussion (*slow* DSL installer at home)
but just wanted to note that the balance I've wanted to strike (if not
perhaps one I've consistently hit) is to be pythonic except in areas
which are so intrinsically Mac-specific that anyone using them will
need to learn a bit of Cocoa anyway - my feeling being that in such
cases it's better to have the terms be Google-able rather than a
Pythonic variant nobody else uses. As an example, if you're dealing
with NSWorkspace notifications, it's reasonable to say that you're
going to need to learn at least a little Cocoa.

Chris