Automating a wxPython application with pywinauto

[Don Dwiggins]

I have a wxPython app, running on Windows 7, that I'd like to automate using pywinauto.  To assist in this process, I installed AutoIt, to get its nice Window Info app.

The first window that the app puts up is a logon dialog, consisting of a combo box and two text boxes (for user name and password).  Window Info shows the structure of this nicely, and I can identify the two text boxes by the (apparently arbitrarily assigned) names Edit2 and Edit3.  However, I'd like to identify them by names that aren't dependent of their position in the parent window.  I notice that WindowInfo shows a "Name" attribute, which is empty for all controls.

I'd like to be able to assign a value to this attribute so that, if the relative position of controls change, the automation script won't break.  I've experimented a bit using Window methods like SetName, SetID, and SetLabel, but the values I set don't show up.

So the question is, is there some magical incantation that I can use to assign an "automatable" name to controls (and maybe other windows).

Thanks for any good words,

--

Don Dwiggins
Advanced Publishing Technology

[Werner]

Hi Don,

"name" and "Set/GetName" should do the trick, I think.

This is used e.g. by wx.lib.agw.persist and you can also see it in action when you use the WIT (http://wiki.wxpython.org/Widget%20Inspection%20Tool), i.e. the following will show "paneText" as the name instead of the default "panel" if one doesn't assign anything or e.g. Boa assigns unique names like "panel1", "panel2" etc etc.

paneText = sc.SizedPanel(self.GetContentsPane(), name='paneText')

Werner

[Steve Barnes]

--
You received this message because you are subscribed to the Google Groups "wxPython-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to wxpython-user...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

If you add a name="SomeName" parameter to the creator for each control it should show up.

--
Steve Gadget Barnes

[Don Dwiggins]

Werner,

Hi Don,

On 15/03/2013 02:47, Don Dwiggins wrote:

{snip}

So the question is, is there some magical incantation that I can use to assign an "automatable" name to controls (and maybe other windows).

"name" and "Set/GetName" should do the trick, I think.

This is used e.g. by wx.lib.agw.persist and you can also see it in action when you use the WIT (http://wiki.wxpython.org/Widget%20Inspection%20Tool), i.e. the following will show "paneText" as the name instead of the default "panel" if one doesn't assign anything or e.g. Boa assigns unique names like "panel1", "panel2" etc etc.

paneText = sc.SizedPanel(self.GetContentsPane(), name='paneText')

Thanks for the reply.  Yes, this "name" works nicely with WIT.  Unfortunately, it doesn't show up at the "Windows level" (whatever the right name is) that Window Info and pywinauto see.

IIRC, WX wraps each "platform window" in an object that wx applications use.  I might need to do something ugly like getting the Windows window handle and operating on that.  Robin?

[Robin Dunn]

Werner wrote:
> Hi Don,
>
> On 15/03/2013 02:47, Don Dwiggins wrote:
>> I have a wxPython app, running on Windows 7, that I'd liketo

>> automateusing pywinauto. To assist in this process, Iinstalled AutoIt,

>> to get its nice Window Info app.
>>
>> The first window that the app puts up is a logon dialog, consisting of

>> a combobox and two text boxes (for user name and password). Window

>> Info shows the structure of this nicely, and I can identify the two
>> text boxes by the (apparently arbitrarily assigned) names Edit2 and
>> Edit3. However, I'd like to identify them by names that aren't
>> dependent of their position in the parent window. I notice that
>> WindowInfo shows a "Name" attribute, which is empty for all controls.
>>
>> I'd like to be able to assign a value to this attribute so that, if
>> the relative position of controls change, the automation script won't
>> break. I've experimented a bit using Window methods like SetName,

>> SetID, and SetLabel, but thevalues I set don't show up.

>>
>> So the question is, is there some magical incantation that I can use
>> to assign an "automatable" name to controls (and maybe other windows).
> "name" and "Set/GetName" should do the trick, I think.
>
> This is used e.g. by wx.lib.agw.persist and you can also see it in
> action when you use the WIT
> (http://wiki.wxpython.org/Widget%20Inspection%20Tool), i.e. the
> following will show "paneText" as the name instead of the default
> "panel" if one doesn't assign anything or e.g. Boa assigns unique names
> like "panel1", "panel2" etc etc.
>
> paneText = sc.SizedPanel(self.GetContentsPane(), name='paneText')
>

Unfortunately that name is only used in the wx layers, it is not passed
down to the native widget, so it would never be shown in things like
autoit's tools that only look at the win32 parts of the window and don't
know about the wx parts. There may be a way to set that name via
PyWin32 or ctypes however, but I don't know which API that is off the
top of my head.

Another way that should work is to use SetLabel. For things like
wx.StaticText or buttons that is the visible label or title, but all
widgets on Windows have one and wxWidgets sets it when you call
SetLabel, even for things like panels. Using the window ID should be
doable too, although you would then have to use fixed IDs in your
application.


--
Robin Dunn
Software Craftsman
http://wxPython.org

[Tim Roberts]

No.  Controls in Windows do not have names.  The names and tags you use in wxPython are merely conventions used internally to Python and wxPython.  They exist within your application code, and nowhere else.

For an arbitrary Win32 application, which is what your app looks like to AutoIt, the only information that is available is the window handle and the dialog control ID.  Most wxPython applications now eschew the ID, which makes things a bit more difficult.  If you do decide to assign a unique ID to your controls, you should be able to access that from AutoIt.

Window automation is simply, by its nature, a very delicate task.  If the dialog you're controlling changes, then your controller script has to change.

-- 
Tim Roberts, ti...@probo.com
Providenza & Boekelheide, Inc.

[Don Dwiggins]

Werner, Robin, Tim,

Thanks for the help.  To document what I've learned here, and by further exploration of my own:

• Per the AutoIt docs,  the "Name" attribute it shows seems only to pertain to .Net controls
• The numeric ID assigned to a wx widget does show up in the ID field of the underlying window.  This is probably the only reliable way to identify a control unambiguously, and invariant to the addition, moving, or removal of other controls in a window.
  I may wind up doing something like creating a sequence of structured widget names (in a module, imported by both the app and pywinauto script), then using the index of each in the sequence as the ID.

Another possibilty occurred to me.  My main motivation for using pywinauto (or other way of automating an application), is to create a "library" for the application testing system RobotFramework (https://code.google.com/p/robotframework/), which supports writing tests in various styles and at various levels, and allows the use of different automation tools.  So, could I adapt the "guts" of the Widget Inspection Tool to turn it into a wx-specific automation tool?  This might be both easier and more powerful than fiddling with platform-level window structures, in that it allows expressing the tests in a notation close to the wxPython "source code".  I'll look into this a bit.

Thanks again,
Don

[Tim Roberts]

Don Dwiggins wrote:

Another possibilty occurred to me.  My main motivation for using pywinauto (or other way of automating an application), is to create a "library" for the application testing system RobotFramework (https://code.google.com/p/robotframework/), which supports writing tests in various styles and at various levels, and allows the use of different automation tools.  So, could I adapt the "guts" of the Widget Inspection Tool to turn it into a wx-specific automation tool?  This might be both easier and more powerful than fiddling with platform-level window structures, in that it allows expressing the tests in a notation close to the wxPython "source code".

Well, you need to be aware that the Widget Inspection Tool is "in-process".  It needs to look at Python object information, and that's only possible  if WIT is running in the same Python interpreter as the wx application.  You can't write an application that uses WIT to inspect objects in another process.

[Don Dwiggins]

On 3/18/13 9:58 AM, Tim Roberts wrote:

Well, you need to be aware that the Widget Inspection Tool is "in-process".  It needs to look at Python object information, and that's only possible  if WIT is running in the same Python interpreter as the wx application.  You can't write an application that uses WIT to inspect objects in another process.

Thanks for that, Tim.  That may or may not be a deal-breaker.

[Robin Dunn]

It could probably be doable though, but it would take a lot of work and
it would require cooperation with the remote process. It would be like
Python debuggers that are able to debug, control, introspect, etc. a
Python application running in a different process.

[Don Dwiggins]

On 3/18/13 12:22 PM, Robin Dunn wrote:

> Tim Roberts wrote:
>>
>> Well, you need to be aware that the Widget Inspection Tool is
>> "in-process". It needs to look at Python object information, and that's
>> only possible if WIT is running in the same Python interpreter as the wx
>> application. You can't write an application that uses WIT to inspect
>> objects in another process.
>
> It could probably be doable though, but it would take a lot of work
> and it would require cooperation with the remote process. It would be
> like Python debuggers that are able to debug, control, introspect,
> etc. a Python application running in a different process.

Here's another thought: create a mixin that hooks into the application
like WIT, but sets up a simple RPC server, like SimpleXMLRPCServer or
the JSON equivalent. I may have to start the application manually, but
after that, it'd be a matter of sending RPC requests to drive the
application and get information. Am I missing any gotcha's with that
approach?

[Chris Barker - NOAA Federal]

>> You can't write an application that uses WIT to inspect
>> objects in another process.
>
>
> It could probably be doable though, but it would take a lot of work and it
> would require cooperation with the remote process. It would be like Python
> debuggers that are able to debug, control, introspect, etc. a Python
> application running in a different process.

If you do want to go this route, you should take a look at what
iPython has done with zeroMQ

-Chris

--

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

Chris....@noaa.gov

[Robin Dunn]

That is basically what I was thinking of too, and is similar to how
Python debuggers usually work. In addition the PyCrust that is running
in the WIT would need to be changed so it is executing the code in the
other process as well. That might be the trickiest part.

[Don Dwiggins]

On 3/19/13 10:49 PM, Robin Dunn wrote:
>
>> Here's another thought: create a mixin that hooks into the application
>> like WIT, but sets up a simple RPC server, like SimpleXMLRPCServer or
>> the JSON equivalent. I may have to start the application manually, but
>> after that, it'd be a matter of sending RPC requests to drive the
>> application and get information. Am I missing any gotcha's with that
>> approach?
> That is basically what I was thinking of too, and is similar to how
> Python debuggers usually work. In addition the PyCrust that is
> running in the WIT would need to be changed so it is executing the
> code in the other process as well. That might be the trickiest part.
>

Hmm... The way I was envisioning it, the WIT (actually, the automation
"driver") would be the client, and the server would be "in-process" with
the application being automated. What "calls" are recognized by the
server depends on what functions of the application need to be
automated, and what "level" the automation works at.

For example, filling in and saving a form in a dialog box could involve
a whole sequence of RPC calls, to bring up the dialog, fill in fields,
select choices, and finally click Save -- or it could be implemented as
a single call, saying "fill in form X using this data" -- or both, for
that matter.