Concurrent pyinstaller builds

34 views
Skip to first unread message

Niemann, Hartmut

unread,
Mar 9, 2020, 6:15:30 AM3/9/20
to pyins...@googlegroups.com

Hello!

 

pyinstaller (3.2.1 on python 2.7) has a problem when running concurrently, which happens now and then on our Jenkins build system:

 

15217 INFO: Redirecting Microsoft.VC90.CRT version (9, 0, 21022, 8) -> (9, 0, 30729, 9518)

15217 INFO: Redirecting Microsoft.VC90.MFC version (9, 0, 21022, 8) -> (9, 0, 30729, 1)

15219 INFO: Updating manifest in C:\WINDOWS\system32\config\systemprofile\AppData\Roaming\pyinstaller\bincache00_py27_32bit\win32ui.pyd

15221 INFO: Updating resource type 24 name 2 language 1033

15358 ERROR: C:\WINDOWS\system32\config\systemprofile\AppData\Roaming\pyinstaller\bincache00_py27_32bit\unicodedata.pyd

Traceback (most recent call last):

  File "C:\DEVTOOLS\Python\27-32\Scripts\pyinstaller-script.py", line 11, in <module>

    load_entry_point('PyInstaller==3.2.1', 'console_scripts', 'pyinstaller')()

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\__main__.py", line 90, in run

    run_build(pyi_config, spec_file, **vars(args))

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\__main__.py", line 46, in run_build

    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\build_main.py", line 788, in main

    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\build_main.py", line 734, in build

    exec(text, spec_namespace)

  File "<string>", line 37, in <module>

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\api.py", line 402, in __init__

    strip_binaries=self.strip, upx_binaries=self.upx,

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\api.py", line 191, in __init__

    self.__postinit__()

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\datastruct.py", line 161, in __postinit__

    self.assemble()

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\api.py", line 254, in assemble

    dist_nm=inm)

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\building\utils.py", line 268, in checkCache

    res = winmanifest.GetManifestResources(os.path.abspath(cachedfile))

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\utils\win32\winmanifest.py", line 1005, in GetManifestResources

    return winresource.GetResources(filename, [RT_MANIFEST], names, languages)

  File "c:\devtools\python\27-32\lib\site-packages\PyInstaller\utils\win32\winresource.py", line 171, in GetResources

    hsrc = win32api.LoadLibraryEx(filename, 0, LOAD_LIBRARY_AS_DATAFILE)

pywintypes.error: (32, 'LoadLibraryEx', 'The process cannot access the file because it is being used by another process.')

 

What can I do about that?

 

With best regards

Hartmut Niemann

Eric Fahlgren

unread,
Mar 9, 2020, 12:26:34 PM3/9/20
to pyins...@googlegroups.com
The best solution right now appears to be "re-run the build..."

https://github.com/pyinstaller/pyinstaller/issues/3809

I've tried to find the root cause, but it's so spurious as to be impossible.  If you can find a way to reliably reproduce the issue (I have not been able to), I'm sure someone could figure it out.


--
You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyinstaller...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyinstaller/AM6PR10MB2568AD79AEF295CDC2A0F974FCFE0%40AM6PR10MB2568.EURPRD10.PROD.OUTLOOK.COM.

Niemann, Hartmut

unread,
Mar 27, 2020, 2:15:30 AM3/27/20
to pyins...@googlegroups.com

Hello!

 

I might have found a solution to this.

 

set PYINSTALLER_CONFIG_DIR=%WORKSPACE%

 

inside each the Jenkins job gives each pyinstaller process its own bincache dir.

 

Hartmut, would that be a valid solution, or are there implications I am not aware of?

 

(Hartmut, regarding your PM in this thread: I have asked, and I am still working on finding some funding inside our company.)

 

Stay healthy!

 

Hartmut.

Hartmut Goebel

unread,
Apr 4, 2020, 5:30:27 AM4/4/20
to pyins...@googlegroups.com
Hi, Hartmut:

I might have found a solution to this.

set PYINSTALLER_CONFIG_DIR=%WORKSPACE%

inside each the Jenkins job gives each pyinstaller process its own bincache dir.

Hartmut, would that be a valid solution, or are there implications I am not aware of?

This should work since the error seems to be caused by concurrent access to the cached files and this changes the location of the cache.

It also seems to be the best solution, as it works the same for all platforms. (PYINSTALLER_CONFIG_DIR overwrite any platform specific paths).

Thanks for pointing to this - I was not aware of this option anymore :-)

--
Schönen Gruß
Hartmut Goebel
Dipl.-Informatiker (univ), CISSP, CSSLP, ISO 27001 Lead Implementer
Information Security Management, Security Governance, Secure Software Development

Goebel Consult, Landshut
http://www.goebel-consult.de

Blog: https://www.goe-con.de/blog/deshalb-tr-069-ausschalten
Kolumne: https://www.goe-con.de/hartmut-goebel/cissp-gefluester/2011-10-aus-der-schublade-in-die-koepf

0x7B752811BF773B65.asc
Reply all
Reply to author
Forward
0 new messages