Pyinstaller create this error :“File ”multiprocessing\connection.py“, line 691, in PipeClient”

[Centre de Formation France]

Hi everyone,

I am trying to make the executable of my python script with Pyinstaller. I am using:

PyInstaller: 3.6

Python: 3.7.7

Windows 10 64Bits

When I run my script from Windows command terminal, it works fine. When I run the exe produced by Pyinstaller I have this error shown on terminal:

Process SpawnProcess-4:
Traceback (most recent call last):
  File "multiprocessing\process.py", line 297, in _bootstrap
  File "multiprocessing\process.py", line 99, in run
  File "concurrent\futures\process.py", line 233, in _process_worker
  File "multiprocessing\queues.py", line 113, in get
  File "multiprocessing\managers.py", line 943, in RebuildProxy
  File "multiprocessing\managers.py", line 793, in __init__
  File "multiprocessing\managers.py", line 847, in _incref
  File "multiprocessing\connection.py", line 490, in Client
  File "multiprocessing\connection.py", line 691, in PipeClient
FileNotFoundError: [WinError 2] The system cannot find the file specified

A process in the process pool was terminated abruptly while the future was running or pending. --> Error multiprocessor

This is my piece of code which create this error:

 with concurrent.futures.ProcessPoolExecutor() as executor:
            try:
                #multiprocesses = executor.map(mymodules.run_smartphone, list_arguments_smartphones)
                multiprocesses = executor.map(mymodules.run_smartphone, list_smartphones_connected,
                                              [lock] * len(list_smartphones_connected))
                for function_return_value in multiprocesses:
                    print(function_return_value)

            except Exception as ex:
                print(f"{ex} --> Error multiprocesses")

So I made the exe with the option --dir in order to make the executable with dolls, and I get exact same issue.

I run some specific Pyinstaller commands, it is displaying something, but I don't understand what it means:

E:\>pyi-bindepend Myprogram.exe
Myprogram.exe {'KERNEL32.dll', '*_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.0.0_none', 'WS2_32.dll', 'ADVAPI32.dll'}

E:\>pyi-archive_viewer Myprogram.exe
 pos, length, uncompressed, iscompressed, type, name
[(0, 270, 354, 1, 'm', 'struct'),
 (270, 1124, 1832, 1, 'm', 'pyimod01_os_path'),
 (1394, 4376, 9337, 1, 'm', 'pyimod02_archive'),
 (5770, 7417, 18651, 1, 'm', 'pyimod03_importers'),
 (13187, 1868, 4175, 1, 's', 'pyiboot01_bootstrap'),
 (15055, 266, 321, 1, 's', 'pyi_rth_certifi'),
 (15321, 1080, 1775, 1, 's', 'pyi_rth_multiprocessing'),
 (16401, 8845, 18589, 1, 's', 'Myprogram'),
 (25246, 3052666, 3052666, 0, 'z', 'PYZ-00.pyz')]
?

so I added in the .spec file some hiddenimport:

hiddenimports=["socks","multiprocessing","concurrent"],

as I saw issues with multiprocessing and futures.. But it didn't fix anything

I tried this one:

hiddenimports=["socks","multiprocessing","concurrent.futures"],

still same issue.

So I check this line 691 in this connection.py file, but I don't understand the issue:

def PipeClient(address):
        '''
        Return a connection object connected to the pipe given by `address`
        '''
        t = _init_timeout()
        while 1:
            try:
                _winapi.WaitNamedPipe(address, 1000)<==------------------------[line 691]
                h = _winapi.CreateFile(
                    address, _winapi.GENERIC_READ | _winapi.GENERIC_WRITE,
                    0, _winapi.NULL, _winapi.OPEN_EXISTING,
                    _winapi.FILE_FLAG_OVERLAPPED, _winapi.NULL
                    )
            except OSError as e:
                if e.winerror not in (_winapi.ERROR_SEM_TIMEOUT,
                                      _winapi.ERROR_PIPE_BUSY) or _check_timeout(t):
                    raise
            else:
                break
        else:
            raise

        _winapi.SetNamedPipeHandleState(
            h, _winapi.PIPE_READMODE_MESSAGE, None, None
            )
        return PipeConnection(h)

I am not good enough in Python to fix this issue. I can just make the conclusion that Pyinstaller is breaking something as the script is running perfectly fine from the terminal. Also I see that system is saying it cannot find a file. So a file or module is missing, that is why I tried some stuff in .spec file.

Does anyone know the source of my problem please?

[bwoodsend]

Multiproccessing requires special support to freeze. Think link is probably all you're missing.

https://kite.com/python/docs/multiprocessing.freeze_support

Brénainn

[Centre de Formation France]

Thank you so much for trying to help me.

I did add this freeze part:

if __name__ == '__main__':

    import multiprocessing
    multiprocessing.freeze_support()

So I don't think it is the issue.

[bwoodsend]

Ok your doing the freeze_support() correctly. So it's not that.

I mix PyInstaller with multiprocessing quite often and it's always been fine so it should be possible. I hadn't heard of concurrent but it just tested PyInstaller on this example from their docs and it also worked fine. It looks functionally equivalent to the snippet you posted. And I'm using the same versions of Python and PyInstaller so it doesn't look like it's a difference in versions.

Also I see that system is saying it cannot find a file. So a file or module is missing, that is why I tried some stuff in .spec file.

I'm fairly certain that when it says file it means a pipe connection rather than a physical file on your hard disk. So playing with the hiddenimports option is unlikely to do any good.

Could you try freezing the docs example and see if that works?

Brénainn

[Centre de Formation France]

Thank you. I’ll try that asap.