Suds hangs in packaged app

[Glenn Ramsey]

Hi,

I have an app where I am calling a web service asynchronously in a thread using
suds and concurrent.futures. On Windows (7) it hangs in the suds.client.Client
constructor when the app is packaged with PyInstaller. It also hangs if I use
threading.Thread instead of futures.

It works properly on OSX and when run from the command line or from
Eclipse/PyDev in Windows.

Does anyone know what the problem might be? How can I debug this in the packaged
app?

Glenn

[Glenn Ramsey]

Some further info. It only hangs if PyQt5 (5.2.1) is included in the package.

If I add 'import PyQt5.Qt' to the script, but don't use PyQt for anything, then
that causes the hang. Without PyQt5, it runs correctly.

Glenn

[Martin Zibricky]

On Friday 13 of June 2014 12:23:36 Glenn Ramsey wrote:
> It works properly on OSX and when run from the command line or from
> Eclipse/PyDev in Windows.

Could it be that the windows version uses --onefile?

[Martin Zibricky]

On Friday 13 of June 2014 12:23:36 Glenn Ramsey wrote:

> It works properly on OSX and when run from the command line or from
> Eclipse/PyDev in Windows.
>
> Does anyone know what the problem might be? How can I debug this in the
> packaged app?

You haven't mentioned pyinstaller version.

[Glenn Ramsey]

Oops, sorry. I'm using the latest develop branch.

Also just to be clear about what is hanging. The main thread keeps running, it's
the thread that is calling suds that hangs.

Glenn

[Glenn Ramsey]

Here is a minimal example to demonstrate the problem.

What tools can I use to debug this?

sudshang.py:

import sys
import suds
import suds.client
#import PyQt5.Qt # uncomment this to cause hang

import threading
from functools import partial

WebServiceTimeout = 30
url = "http://www.webservicex.net/CurrencyConvertor.asmx?WSDL"

def get_with_cb(callback):
print "getting %s" % url
# hang happens on the following line
client = suds.client.Client(url, timeout=WebServiceTimeout)
print "client is: ", client
callback("get_with_cb")

def callback(arg):
print "done", arg
sys.stdout.flush()

class Async(object):
def __init__(self, callback):
self._callback = callback

def start(self):
p = partial(get_with_cb, self._callback)
t = threading.Thread(target=p)
t.start()
sys.stdout.flush()
print "thread started"
self.t = t

def join(self):
self.t.join()

def main():
# call it in the main thread to prove that it works
get_with_cb(callback)
# call it in another thread
w = Async(callback)
w.start()
print "waiting"
w.join()
print "finished waiting"

if __name__ == "__main__":
main()

-----

Freeze using:

pyinstaller.py -y sudshang[.py|.spec]

[Glenn Ramsey]

On 13/06/14 20:05, Martin Zibricky wrote:

No, it uses the default one folder option.

[Glenn Ramsey]

I have narrowed this down further. It also occurs on OSX and not Windows as I
had first thought.

There is a new file: loader/rthooks/pyi_rth_qt5.py which causes all of the Qt
modules to get included in the package if one module is included. It also merges
the names from each module into the PyQt5.Qt namespace. So if I import only
PyQt.QtCore then all of the Qt modules get included in the package.

So one of the Qt modules that I am not using is interfering somehow with suds.
I'll keep trying to track it down.

I have worked around my problem by disabling pyi_rth_qt5.py in rthooks.dat. So
that only the Qt modules that I am using get included.

There appears also to be a related bug - when any Qt module is imported you have
to also import PyQt5.Qt or there is an error about pyi_rth_qt5.py not being able
to find the Qt module when running the packaged script.

Glenn

[Martin Zibricky]

On Tuesday 17 of June 2014 15:01:54 Glenn Ramsey wrote:
> There is a new file: loader/rthooks/pyi_rth_qt5.py which causes all of the
> Qt modules to get included in the package if one module is included. It
> also merges the names from each module into the PyQt5.Qt namespace. So if I
> import only PyQt.QtCore then all of the Qt modules get included in the
> package.

Recently there was a pull request that added pyi_rth_qt5.py because of a bug
in PyQt5 or Qt5.

Could it be related?

[Martin Zibricky]

On Tuesday 17 of June 2014 15:01:54 Glenn Ramsey wrote:

> So one of the Qt modules that I am not using is interfering somehow with
> suds. I'll keep trying to track it down.

Could it be that PyQt5 somehow does not work properly with python threads?

[Glenn Ramsey]

Yes, as I said above, the error I had was indirectly caused by this change
because it causes all of the Qt modules, including those that are not used, to
get packaged.

As an aside, I think this change needs some refinement. It may help
inexperienced developers by ensuring that their packaged PyQt program will run,
but there is also the other case where you don't want 10's of MB of unused
libraries included in the package. What would be the right way to handle this? I
guess I could explicitly remove the unwanted libraries in my spec file.

Glenn

[Glenn Ramsey]

It works in a non-packaged app, and also it works if I only the include in the
package the PyQt modules that my program uses.