Controlling the Mac OSX GUI via Python?

[maitlandm...@gmail.com]

I'd like to use Python to do some simple OSX GUI manipulations such as automatically arranging windows, periodically refreshing applications across multiple desktops, etc.. Is this possible and if so, is there a preferred package?

I know that Qt, Tkinter, and such are available for heavy-duty custom work but I think these are intended to create UIs from the ground up vs. the maintenance work I'd like to do.

I have tried to learn Applescript and Automator but they aren't really scratching my itch. I am more comfortable with Python. Seeking FMs to RT, so to speak.

[Terry Reedy]

On 6/30/2016 3:22 PM, maitlandm...@gmail.com wrote:
> I'd like to use Python to do some simple OSX GUI manipulations such
> as automatically arranging windows, periodically refreshing
> applications across multiple desktops, etc.. Is this possible and if
> so, is there a preferred package?

Cpython comes with about 16 *nix-specifio modules, 4 Window-specific
modules, and none for Mac. People doing serious Windows-specific work
usually download other packages. Unless you can do everything you want
by calling system utilities via subprocess, you will have to to the same.

> Seeking FMs to RT, so to speak.

????

--
Terry Jan Reedy

[Chris Angelico]

On Fri, Jul 1, 2016 at 6:30 AM, Terry Reedy <tjr...@udel.edu> wrote:
>> Seeking FMs to RT, so to speak.
>
>

> ????

Derivative of the old expression, "RTFM". Normally it means "Go read
the manual that you ought to have already read", but AIUI, the OP is
saying that he'd be happy to be pointed to documentation somewhere.

ChrisA

[hasan...@gmail.com]

>On 6/30/2016 3:22 PM, you wrote:
>> I'd like to use Python to do some simple OSX GUI manipulations such
>> as automatically arranging windows, periodically refreshing
>> applications across multiple desktops, etc.. Is this possible and if
>> so, is there a preferred package?

Please see this page https://docs.python.org/2/using/mac.html. The following will show a window, entitled "Hello world":


#!/pyobjc/virtualenv/path/bin/python

import sys
import os

from AppKit import *
from PyObjCTools import AppHelper


class Delegate (NSObject):
def applicationDidFinishLaunching_(self, aNotification):
'''Called automatically when the application has launched'''
print "Hello, World!"

def windowWillClose_(self, aNotification):
'''Called automatically when the window is closed'''
print "Window has been closed"
NSApp().terminate_(self)


def main():
a=NSApplication.sharedApplication()
delegate = Delegate.alloc().init()
a.setDelegate_(delegate)
frame = ((200.0, 300.0), (250.0, 100.0))
w = NSWindow.alloc().initWithContentRect_styleMask_backing_defer_(frame, 15, 2, 0)
w.setDelegate_(delegate)
w.setTitle_(u'Hello, World!')
w.orderFrontRegardless()
AppHelper.runEventLoop()

if __name__ == '__main__':
main()

[Lawrence D’Oliveiro]

On Friday, July 1, 2016 at 7:22:43 AM UTC+12, Maitland Montmorency wrote:
> I'd like to use Python to do some simple OSX GUI manipulations such as
> automatically arranging windows, periodically refreshing applications
> across multiple desktops, etc..
>

> I have tried to learn Applescript and Automator but they aren't really
> scratching my itch.

GUIs, by their nature, are not designed to be automated.

[Chris Angelico]

Doesn't mean they can't be! Ultimately, GUIs are just code, same as
everything else is, and code can be automated. On my Linux box (Debian
with Xfce), I've automated several aspects of the GUI, including
making certain windows "sticky" (ie visible regardless of which
workspace I'm on), and displaying crucial information in appropriate
places.

That said, though, there aren't always Python bindings for these sorts
of facilities. I end up calling on subprocesses most of the time,
because that's the easiest way to do things.

ChrisA

[Lawrence D’Oliveiro]

On Friday, July 1, 2016 at 10:35:42 AM UTC+12, Chris Angelico wrote:

>
> On Fri, Jul 1, 2016 at 8:27 AM, Lawrence D’Oliveiro wrote:
>>
>> GUIs, by their nature, are not designed to be automated.
>
> Doesn't mean they can't be!

It’s more trouble than it’s worth. Different (even minor) versions of the program will move UI elements around. Things that look and behave like buttons might have underlying code that is not quite the same as that for buttons. There will be subtle timing issues, where something doesn’t quite get enabled in time for your script to click on it. You will suffer mysterious intermittent problems, where a sequence works one time but not another time.

Again: GUIs are for manual operations to be performed by humans. For automation, you need a function-based scripting API and a command line.

[Maitland Montmorency]

On Thursday, June 30, 2016 at 3:35:42 PM UTC-7, Chris Angelico wrote:
> On Fri, Jul 1, 2016 at 8:27 AM, Lawrence D’Oliveiro
> <lawren...@gmail.com> wrote:
> > On Friday, July 1, 2016 at 7:22:43 AM UTC+12, Maitland Montmorency wrote:
> >> I'd like to use Python to do some simple OSX GUI manipulations such as
> >> automatically arranging windows, periodically refreshing applications
> >> across multiple desktops, etc..

> >...

> > GUIs, by their nature, are not designed to be automated.
>

> Doesn't mean they can't be! Ultimately, GUIs are just code, ...
> ChrisA


Thanks, everyone! I'll go read about PyObjC. I'm not sure if I have the chops to take advantage of it but it looks like the bridge into Apple's development environment I'm seeking.

GUI automation: Good catch on the wording. I perhaps should have said GUI element manipulation instead. I like my Mac a great deal but there are some aspects of the Windows interface that I miss; e.g. tile or cascade all windows with a single click. So, I thought I would try my hand at doing it through Python vs. using canned Applescripts.

>>> Seeking FMs to RT, so to speak.

Chris A interpreted it perfectly. Often the hardest part of the problem is knowing how to ask for the manual whose name you don't even know...

-MM

[Christian Gollwitzer]

Am 01.07.16 um 01:12 schrieb Lawrence D’Oliveiro:

> On Friday, July 1, 2016 at 10:35:42 AM UTC+12, Chris Angelico wrote:
>>
>> On Fri, Jul 1, 2016 at 8:27 AM, Lawrence D’Oliveiro wrote:
>>>
>>> GUIs, by their nature, are not designed to be automated.
>>
>> Doesn't mean they can't be!
>
> It’s more trouble than it’s worth. Different (even minor) versions of
> the program will move UI elements around. Things that look and behave
> like buttons might have underlying code that is not quite the same as
> that for buttons. There will be subtle timing issues, where something
> doesn’t quite get enabled in time for your script to click on it. You
> will suffer mysterious intermittent problems, where a sequence works
> one time but not another time.

I think you misunderstood the request. Yes, simulating mouse clicks with
fixed coordinates etc. is prone to such failure, but there are better
methods. These mouse clicks and keyboard events usually trigger a method
call inside the GUI program. If there are any means to call that method
directly, you are independent from the graphics itself.

These bindings then usually call some higher-level functions. Many
applications export their internal structure via some general
communications mechanism. On Windows, COM is such a mechanism, CORBA is
another one, and on the Mac that are Apple Events. AppleScript is a
Mac-specific (strange) language which allows you to send arbitrary Apple
events to applications. This has nothing to do with simulating mouse
clicks. A deprecated Python package is found here:

http://appscript.sourceforge.net/py-appscript/

If you look at the first example:
app('TextEdit').documents['ReadMe'].paragraphs[1].get()

its immediately clear that this is on a very different level than
simulating Copy/Paste using the mouse to get the first paragraph.
According to my Google-Fu, howver, py-appscript stopped working with
some OSX update. One possibility would be to send the equivalent
applescript to the "osascript" program via subprocess.

> Again: GUIs are for manual operations to be performed by humans. For
> automation, you need a function-based scripting API and a command
> line.

Yes and no. Sometimes GUIs export an API which is just not obvious to
the user, sometimes they have an internal structure which can be exploited.

Christian

[Lawrence D’Oliveiro]

On Friday, July 1, 2016 at 4:59:11 PM UTC+12, Christian Gollwitzer wrote:
> Yes, simulating mouse clicks with
> fixed coordinates etc. is prone to such failure, but there are better
> methods. These mouse clicks and keyboard events usually trigger a method
> call inside the GUI program. If there are any means to call that method
> directly, you are independent from the graphics itself.

Even if you could get it working reliably, scripting a GUI is going to be slow.

GUIs are only designed to work at human speeds, after all.

[Christian Gollwitzer]

Am 02.07.16 um 05:16 schrieb Lawrence D’Oliveiro:

It might be slow, but look at his request in the OP "automatically

arranging windows, periodically refreshing applications across multiple

desktops". A button which arranges three programs in a row could be
useful for some users. Or an automatic reload of a status website inside
the browser, where you can't change the website to embed autorefresh
code. I've got a tool running which can remap special keys (ctrl, fn) to
ordinary chars. Also sometimes a complex functionality is buried in a
GUI. Consider a server converting MS Word files into PDF. The only
method which reliably works is COM to drive MS Office, yes it's clumsy,
but unless you persuade MS to release Office as a Python library you
have no other option.

Speed is not of any concern here. If execution speed would be important,
why use Python at all?

Christian