Subprocess & PyInstaller

[Benjam901]

Hello all,

It is time to finally get a project out as open source but I am having a serious roadblock issue with subprocess.

- In the code I create a new QThread for the subprocess function to run on.

- Once the QThread is running I use subprocess.run() OR subprocess.Popen (whatever works) to call ffmpeg to convert some audio files.

This works within PyCharm HOWEVER as soon as I package it up into a .app for mac it crashes immediately upon trying to run the subprocess without any errors or stacktrace.

Has anyone had any issues like this before, I would really like this project to be finished and this is the last blocker

Cheers, 

Ben

[Justin Israel]

You are developing on a mac as well and the only difference is source vs app packaging?

Is the crash a segfault?

Is the path to ffmpeg an absolute path or is it relying on $PATH which may be different in the app?

Can you successfully run a subprocess command in the packaged app without a QThread (is the QThread the factor)?

Have you tried QProcess?

Cheers, 

Ben

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/672ff5b3-ceac-4d42-acaf-c448ea375013n%40googlegroups.com.

[Benjam901]

- I am unsure of the major differences but yes I need it compiled to a .app instead of a .exe

- The crash itself I suspect to be a segfault, previously for unknown reasons my IDE was crashing with a SIGABRT error.

- The path to ffmpeg is absolute and uses QStandardPaths to locate the documents folder where ffmpeg lives.

- I have unthreaded the process and it is still the same issue, I will try additionally ro run a subprocess to open terminal or some other native app.

- QProcess I came across recently, what is the major difference between QProcess & QThread? Is it the same difference as Threading & Multiprocessing?

I get this error weirdly irregularly and is hard to repro 100%. It pops up and then goes away...

QThread: Destroyed while thread is still running

Fatal Python error: Aborted

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

// Ben

[Justin Israel]

On Thu, Sep 30, 2021 at 9:13 AM Benjam901 <benandr...@gmail.com> wrote:

- I am unsure of the major differences but yes I need it compiled to a .app instead of a .exe

- The crash itself I suspect to be a segfault, previously for unknown reasons my IDE was crashing with a SIGABRT error.

I've seen this kind of crash when something in python raises an exception in certain Qt contexts like in a signal handler, or somewhere async where the Qt stack freaks out from the python exception.

 

- The path to ffmpeg is absolute and uses QStandardPaths to locate the documents folder where ffmpeg lives.

- I have unthreaded the process and it is still the same issue, I will try additionally ro run a subprocess to open terminal or some other native app.

That is a great debugging point. So then I would focus further on debugging without a QThread involved since it seems to have no impact on the crash.

 

- QProcess I came across recently, what is the major difference between QProcess & QThread? Is it the same difference as Threading & Multiprocessing?

In this case I don't think its a matter of threading + multiprocessing since you are using subprocess to exec. We have seen python/qt specific bugs around that though. 

You could try QProcess to at least isolate subprocess as the problem, since it would be using an entirely different way to launch the child process.

 

I get this error weirdly irregularly and is hard to repro 100%. It pops up and then goes away...

QThread: Destroyed while thread is still running

Fatal Python error: Aborted

Process finished with exit code 134 (interrupted by signal 6: SIGABRT)

I think that is saying that your python QThread object reference is being destroyed, before the underlying system thread has finished. Are you keeping a reference to it while it is still running either by holding the python object, or by having it parented to a Qt object?

 

// Ben

On Wednesday, 29 September 2021 at 19:30:32 UTC+2 justin...@gmail.com wrote:

On Thu, Sep 30, 2021 at 4:34 AM Benjam901 <benandr...@gmail.com> wrote:

Hello all,

It is time to finally get a project out as open source but I am having a serious roadblock issue with subprocess.

- In the code I create a new QThread for the subprocess function to run on.

- Once the QThread is running I use subprocess.run() OR subprocess.Popen (whatever works) to call ffmpeg to convert some audio files.

This works within PyCharm HOWEVER as soon as I package it up into a .app for mac it crashes immediately upon trying to run the subprocess without any errors or stacktrace.

Has anyone had any issues like this before, I would really like this project to be finished and this is the last blocker

You are developing on a mac as well and the only difference is source vs app packaging?

Is the crash a segfault?

Is the path to ffmpeg an absolute path or is it relying on $PATH which may be different in the app?

Can you successfully run a subprocess command in the packaged app without a QThread (is the QThread the factor)?

Have you tried QProcess?

Cheers, 

Ben

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/672ff5b3-ceac-4d42-acaf-c448ea375013n%40googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Python Programming for Autodesk Maya" group.
To unsubscribe from this group and stop receiving emails from it, send an email to python_inside_m...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/ee782b0c-7cae-4807-8d8b-5363660f5328n%40googlegroups.com.

[Chad Vernon]

I think I had similar issues on Windows. I used this to sort them out: https://github.com/pyinstaller/pyinstaller/wiki/Recipe-subprocess

It's written for Windows but maybe you can adapt it to work for Mac

[Benjam901]

I think that is saying that your python QThread object reference is being destroyed, before the underlying system thread has finished. Are you keeping a reference to it while it is still running either by holding the python object, or by having it parented to a Qt object?

Yes, I keep a reference to it in the main thread as a class variable.

It... spookily seems to have resolved itself for now. Which is concerning to say the least. I have had this happen before I believe and it just magically worked on its own accord.

The tool I am writing is cross platform so the above comment will be useful for sure!

[Chris Granados- Xian]

About the QThread, try emiting a custom Signal (Signals are threadsafe) from your secondary thread when whatever process finishes running. Then connect to it from your main thread and, for example, before closing the main thread you could check if a secondary thread is still running and proceed according to whatever behaviour you want.

In my case I have it in the closeEvent of a main dialog. It simply waits for the workers to finish (I use workers) and only then closes the dialog.

To view this discussion on the web visit https://groups.google.com/d/msgid/python_inside_maya/a48e6feb-c33e-4fa0-b1f0-fc442fa2842fn%40googlegroups.com.

--

CHRIS GRANADOS - Xian

Character Animator- CG Supervisor
www.chrisgranados.com
Cel.: +(54911) 40640097
Tel.: +(5411)  45460877
Bs. As., Argentina