[PATCH 0 of 3] PyQt5 patches for Win64 binaries

132 views
Skip to first unread message

Steve Borho

unread,
Sep 7, 2017, 3:36:16 PM9/7/17
to thg...@googlegroups.com
Good news and bad news

I decided to try to build kdiff3 for Qt5 myself, and soon after booting my dual-boot machine into Windows 10 (for the first time in a year) it applied some service packs which wiped out my Linux partition, including my only copy of the VM that was previously building the TortoiseHg installers. Poof.

So I've spent the last two weeks chasing Qt5, PyQt5, QSci, and kdiff3 and after much wrangling (many prereq packages have mostly abandoned Python2) the good news is I have an installer for people to test, which includes Qt 5.9.1 and Kdiff3 compiled against that Qt 5.9.1. https://bitbucket.org/tortoisehg/thg-winbuild/downloads/tortoisehg-dev-4.3.1237.2101-x64.msi

(note; the hg hash used in this installer is a local tag commit that I had to add hg to work around a latesttag bug)

The bad news; I have certainly broken the win32 PyQt4 binary installer process and since I have no 32bit machine on which to test new packages I am considerably unmotivated to try and resurrect the process. I don't think I'll be producing any more x86/win32 packages.

The process for building the python27\lib\site-packages\PyQt5 folder is mostly insane, now that python2 binary PyQt packages are no longer built. This took a great deal of trial and error.

I'm looking for testing feedback for this MSI - one problem I've already noted is that kdiff3 is not found until I open the settings tool and force it to scan for merge tools. I can't explain this. Also I am having to re-key my code signing certificate, I expect it to take another day or so, so this package is unsigned.

Steve Borho

unread,
Sep 7, 2017, 3:36:17 PM9/7/17
to thg...@googlegroups.com
# HG changeset patch
# User Steve Borho <st...@borho.org>
# Date 1504638314 18000
# Tue Sep 05 14:05:14 2017 -0500
# Node ID be7f70bebaabe120760887cc9c7b1f76ca03fca6
# Parent edebbb388e1eed091dc71f6b6836341181c86abf
prep work for Qt5 x64 binary installers

diff -r edebbb388e1e -r be7f70bebaab setup.py
--- a/setup.py Sun Aug 27 17:04:08 2017 +0900
+++ b/setup.py Tue Sep 05 14:05:14 2017 -0500
@@ -352,7 +352,7 @@
rcc = {'PyQt4': 'pyrcc4', 'PyQt5': 'pyrcc5'}[QT_API]
except KeyError:
raise RuntimeError('unsupported Qt API: %s' % QT_API)
- if os.name != 'nt':
+ if os.name != 'nt' or QT_API == 'PyQt5':
return rcc
mod = __import__(QT_API, globals(), locals())
return os.path.join(os.path.dirname(mod.__file__), rcc)
@@ -398,14 +398,25 @@

def _build_translations(self, basepath):
"""Build translations_rc.py which inclues qt_xx.qm"""
- if os.name == 'nt':
- import PyQt4
- trpath = os.path.join(
- os.path.dirname(PyQt4.__file__), 'translations')
+ from tortoisehg.hgqt.qtcore import QT_API
+ if QT_API == 'PyQt4':
+ if os.name == 'nt':
+ import PyQt4
+ trpath = os.path.join(
+ os.path.dirname(PyQt4.__file__), 'translations')
+ else:
+ from PyQt4.QtCore import QLibraryInfo
+ trpath = unicode(
+ QLibraryInfo.location(QLibraryInfo.TranslationsPath))
else:
- from PyQt4.QtCore import QLibraryInfo
- trpath = unicode(
- QLibraryInfo.location(QLibraryInfo.TranslationsPath))
+ if os.name == 'nt':
+ import PyQt5
+ trpath = os.path.join(
+ os.path.dirname(PyQt5.__file__), 'translations')
+ else:
+ from PyQt5.QtCore import QLibraryInfo
+ trpath = unicode(
+ QLibraryInfo.location(QLibraryInfo.TranslationsPath))
builddir = os.path.join(self.get_finalized_command('build').build_base,
'qt-translations')
self.mkpath(builddir)
@@ -555,7 +566,17 @@
import PyQt4
pluginsdir = os.path.join(os.path.dirname(PyQt4.__file__), 'plugins')
return (subdir, [os.path.join(pluginsdir, subdir, e) for e in dlls])
- _data_files.append(qt4_plugins('imageformats', 'qico4.dll', 'qsvg4.dll'))
+
+ def qt5_plugins(subdir, *dlls):
+ import PyQt5
+ pluginsdir = os.path.join(os.path.dirname(PyQt5.__file__), 'plugins')
+ return (subdir, [os.path.join(pluginsdir, subdir, e) for e in dlls])
+
+ from tortoisehg.hgqt.qtcore import QT_API
+ if QT_API == 'PyQt4':
+ _data_files.append(qt4_plugins('imageformats', 'qico4.dll', 'qsvg4.dll'))
+ else:
+ _data_files.append(qt5_plugins('platforms', 'qwindows.dll'))

# Manually include other modules py2exe can't find by itself.
if 'hgext.highlight' in hgextmods:
diff -r edebbb388e1e -r be7f70bebaab tortoisehg/hgqt/qtapp.py
--- a/tortoisehg/hgqt/qtapp.py Sun Aug 27 17:04:08 2017 +0900
+++ b/tortoisehg/hgqt/qtapp.py Tue Sep 05 14:05:14 2017 -0500
@@ -63,7 +63,8 @@

if os.name == 'nt' and getattr(sys, 'frozen', False):
# load QtSvg4.dll and QtXml4.dll by .pyd, so that imageformats/qsvg4.dll
- # can find them without relying on unreliable PATH variable
+ # can find them without relying on unreliable PATH variable. The filenames
+ # change for PyQt5 but the basic problem remains the same.
_mod = __import__(QT_API, globals(), locals(), ['QtSvg', 'QtXml'])
_mod.QtSvg.__name__, _mod.QtXml.__name__ # no demandimport

diff -r edebbb388e1e -r be7f70bebaab tortoisehg/hgqt/qtcore.py
--- a/tortoisehg/hgqt/qtcore.py Sun Aug 27 17:04:08 2017 +0900
+++ b/tortoisehg/hgqt/qtcore.py Tue Sep 05 14:05:14 2017 -0500
@@ -13,7 +13,7 @@
import sys

def _detectapi():
- candidates = ['PyQt4'] # TODO: ['PyQt5', 'PyQt4']
+ candidates = ['PyQt5', 'PyQt4']
if not getattr(sys, 'frozen', False):
api = os.environ.get('THG_QT_API')
if api:

Steve Borho

unread,
Sep 7, 2017, 3:36:19 PM9/7/17
to thg...@googlegroups.com
# HG changeset patch
# User Steve Borho <st...@borho.org>
# Date 1504713943 18000
# Wed Sep 06 11:05:43 2017 -0500
# Node ID 5bd43068a9af86ffd752724d033a5b1061250742
# Parent be7f70bebaabe120760887cc9c7b1f76ca03fca6
setup: ignore Windows10 API DLLs

diff -r be7f70bebaab -r 5bd43068a9af win32/setup.cfg
--- a/win32/setup.cfg Tue Sep 05 14:05:14 2017 -0500
+++ b/win32/setup.cfg Wed Sep 06 11:05:43 2017 -0500
@@ -1,8 +1,29 @@
[py2exe]
-excludes=pywin, pywin.dialogs, pywin.dialogs.list, PyQt4.uic.port_v3, tcl, tk,
+excludes=pywin, pywin.dialogs, pywin.dialogs.list, tcl, tk,
py2exe, setup, setuptools, distutils
-includes=sip
+includes=sip, PyQt5.QtPrintSupport, PyQt5.QtSvg, PyQt5.QtXml
packages=ctypes, email, hgext, hgext.convert, encodings, tortoisehg.util, tortoisehg.hgqt, iniparse, pygments,
nntplib, dulwich, json, sspi, keyring.backends, sqlite3, hgsubversion, hggit,
- hgext3rd, hgext3rd.evolve, hgext3rd.topic, mercurial.cext
-dll_excludes=mswsock.dll, powrprof.dll, shfolder.dll, credui.dll, msvcp90.dll, secur32.dll, tk85.dll, tcl85.dll
+ hgext3rd, hgext3rd.evolve, hgext3rd.topic, mercurial.cext, mercurial.pure
+dll_excludes=mswsock.dll, powrprof.dll, shfolder.dll, credui.dll, msvcp90.dll, secur32.dll, tk85.dll, tcl85.dll,
+ api-ms-win-core-memory-l1-1-2.dll, api-ms-win-core-libraryloader-l1-2-2.dll,
+ api-ms-win-core-registry-l1-1-0.dll, api-ms-win-core-string-l2-1-0.dll,
+ api-ms-win-core-processenvironment-l1-2-0.dll, api-ms-win-core-string-obsolete-l1-1-0.dll,
+ api-ms-win-core-debug-l1-1-1.dll, api-ms-win-core-localization-obsolete-l1-3-0.dll,
+ api-ms-win-core-sidebyside-l1-1-0.dll, api-ms-win-core-processthreads-l1-1-2.dll,
+ api-ms-win-core-kernel32-legacy-l1-1-1.dll, api-ms-win-core-file-l1-2-1.dll,
+ api-ms-win-core-atoms-l1-1-0.dll, api-ms-win-core-winrt-error-l1-1-1.dll,
+ api-ms-win-security-base-l1-2-0.dll, api-ms-win-core-heap-l2-1-0.dll,
+ api-ms-win-core-rtlsupport-l1-2-0.dll, api-ms-win-core-libraryloader-l1-2-0.dll,
+ api-ms-win-core-localization-l1-2-1.dll, api-ms-win-core-sysinfo-l1-2-1.dll,
+ api-ms-win-core-errorhandling-l1-1-1.dll, api-ms-win-core-shlwapi-obsolete-l1-2-0.dll,
+ api-ms-win-core-heap-l1-2-0.dll, api-ms-win-core-handle-l1-1-0.dll,
+ api-ms-win-core-profile-l1-1-0.dll, api-ms-win-core-string-l1-1-0.dll,
+ api-ms-win-core-synch-l1-2-0.dll, api-ms-win-core-timezone-l1-1-0.dll,
+ api-ms-win-core-util-l1-1-0.dll, api-ms-win-core-delayload-l1-1-1.dll,
+ api-ms-win-crt-convert-l1-1-0.dll, api-ms-win-crt-environment-l1-1-0.dll,
+ api-ms-win-crt-heap-l1-1-0.dll, api-ms-win-crt-locale-l1-1-0.dll,
+ api-ms-win-crt-math-l1-1-0.dll, api-ms-win-crt-runtime-l1-1-0.dll,
+ api-ms-win-crt-stdio-l1-1-0.dll, api-ms-win-crt-string-l1-1-0.dll,
+ api-ms-win-crt-time-l1-1-0.dll, api-ms-win-crt-utility-l1-1-0.dll,
+ api-ms-win-crt-filesystem-l1-1-0.dll

Steve Borho

unread,
Sep 7, 2017, 3:36:20 PM9/7/17
to thg...@googlegroups.com
# HG changeset patch
# User Steve Borho <st...@borho.org>
# Date 1504802013 18000
# Thu Sep 07 11:33:33 2017 -0500
# Node ID 8497626da423eda2c159dd5edc9d5ad816a92b3b
# Parent 5bd43068a9af86ffd752724d033a5b1061250742
wix: new PyQt5 files in dist/ folder

diff -r 5bd43068a9af -r 8497626da423 win32/wix/dist.wxs
--- a/win32/wix/dist.wxs Wed Sep 06 11:05:43 2017 -0500
+++ b/win32/wix/dist.wxs Thu Sep 07 11:33:33 2017 -0500
@@ -16,14 +16,18 @@
<File Name="library.zip" KeyPath="yes" />
<File Name="pythoncom27.dll" />
<File Name="pywintypes27.dll" />
- <File Name="QtCore4.dll" />
- <File Name="QtGui4.dll" />
- <File Name="QtNetwork4.dll" />
- <File Name="QtSvg4.dll" />
- <File Name="QtXml4.dll" />
- <File Name="qscintilla2.dll" />
- <File Name="SSLEAY32.dll" />
- <File Name="LIBEAY32.dll" />
+ <File Name="Qt5Core.dll" />
+ <File Name="Qt5Gui.dll" />
+ <File Name="Qt5Widgets.dll" />
+ <File Name="Qt5Network.dll" />
+ <File Name="Qt5PrintSupport.dll" />
+ <File Name="Qt5Svg.dll" />
+ <File Name="Qt5Xml.dll" />
+ <File Name="qscintilla2_qt5.dll" />
+ <File Name="CRYPT32.dll" />
+ <File Name="VCRUNTIME140.dll" />
+ <File Name="MSVCP140.dll" />
+ <File Name="markupsafe._speedups.pyd" />
<File Name="mercurial.cext.base85.pyd" />
<File Name="mercurial.cext.bdiff.pyd" />
<File Name="mercurial.cext.diffhelpers.pyd" />
@@ -33,12 +37,14 @@
<File Name="mercurial.zstd.pyd" />
<File Name="hgext.fsmonitor.pywatchman.bser.pyd" />
<File Name="pyexpat.pyd" />
- <File Name="PyQt4.Qsci.pyd" />
- <File Name="PyQt4.QtCore.pyd" />
- <File Name="PyQt4.QtGui.pyd" />
- <File Name="PyQt4.QtNetwork.pyd" />
- <File Name="PyQt4.QtSvg.pyd" />
- <File Name="PyQt4.QtXml.pyd" />
+ <File Name="PyQt5.Qsci.pyd" />
+ <File Name="PyQt5.QtCore.pyd" />
+ <File Name="PyQt5.QtGui.pyd" />
+ <File Name="PyQt5.QtWidgets.pyd" />
+ <File Name="PyQt5.QtPrintSupport.pyd" />
+ <File Name="PyQt5.QtNetwork.pyd" />
+ <File Name="PyQt5.QtSvg.pyd" />
+ <File Name="PyQt5.QtXml.pyd" />
<File Name="bz2.pyd" />
<File Name="dulwich._diff_tree.pyd" />
<File Name="dulwich._objects.pyd" />
@@ -61,7 +67,6 @@
<File Name="_elementtree.pyd" />
<File Name="_hashlib.pyd" />
<File Name="_multiprocessing.pyd" />
- <File Name="_testcapi.pyd" />
<File Name="_socket.pyd" />
<File Name="_ssl.pyd" />
<File Name="_win32sysloader.pyd" />
@@ -107,8 +112,15 @@
<DirectoryRef Id="INSTALLDIR">
<Directory Id="imgFmtDir" Name="imageformats" FileSource="$(var.SourceDir)/imageformats">
<Component Id="imageFormats" Guid="$(var.imageformats.guid)" Win64='$(var.IsX64)'>
- <File Name="qico4.dll" KeyPath="yes" />
- <File Name="qsvg4.dll" />
+ <File Name="qico.dll" KeyPath="yes" />
+ <File Name="qsvg.dll" />
+ <File Name="qjpeg.dll" />
+ <File Name="qgif.dll" />
+ <File Name="qicns.dll" />
+ <File Name="qtga.dll" />
+ <File Name="qtiff.dll" />
+ <File Name="qwbmp.dll" />
+ <File Name="qwebp.dll" />
</Component>
</Directory>
</DirectoryRef>

Steve Borho

unread,
Sep 8, 2017, 10:55:28 AM9/8/17
to thg...@googlegroups.com
This kdiff3 path issue appears to have been caused by running thg.exe from a shell that did not have TortoiseHg in the PATH; so it was user-error.


Steve

Yuya Nishihara

unread,
Sep 10, 2017, 9:07:19 AM9/10/17
to thg...@googlegroups.com, Steve Borho
On Thu, 07 Sep 2017 14:35:44 -0500, Steve Borho wrote:
> # HG changeset patch
> # User Steve Borho <st...@borho.org>
> # Date 1504638314 18000
> # Tue Sep 05 14:05:14 2017 -0500
> # Node ID be7f70bebaabe120760887cc9c7b1f76ca03fca6
> # Parent edebbb388e1eed091dc71f6b6836341181c86abf
> prep work for Qt5 x64 binary installers

Queued this, thanks.

The next patch couldn't apply probably because of line ending issue. Can you
push the remainder?

André Sintzoff

unread,
Sep 11, 2017, 3:42:56 AM9/11/17
to thg...@googlegroups.com
2017-09-07 21:35 GMT+02:00 Steve Borho <st...@borho.org>:

> So I've spent the last two weeks chasing Qt5, PyQt5, QSci, and kdiff3 and after much wrangling (many prereq packages have mostly abandoned Python2) the good news is I have an installer for people to test, which includes Qt 5.9.1 and Kdiff3 compiled against that Qt 5.9.1. https://bitbucket.org/tortoisehg/thg-winbuild/downloads/tortoisehg-dev-4.3.1237.2101-x64.msi
dev+uns...@googlegroups.com.

I have installed the msi on a Windows 7 machine.
Unfortunately, I'm not able to launch thg.
I got the following error message:

This application failed to start because it could not find or load the
Qt platform plugin "windows" in "".

Reinstalling the application may fix this problem.


After reinstallation, the message is still there.

Any thought?

Thanks

André

Steve Borho

unread,
Sep 12, 2017, 1:15:44 PM9/12/17
to thg...@googlegroups.com
Thanks for testing

I somehow missed the platform folder in the WIX code; I've pushed
fixes for this and for kdiff3 plugin loading and rebuilt kdiff3.exe so
it does not open a console.

new MSI at https://bitbucket.org/tortoisehg/thg-winbuild/downloads/tortoisehg-dev-4.3.1242.1562-x64.msi

--
Steve Borho

André Sintzoff

unread,
Sep 13, 2017, 3:55:20 AM9/13/17
to thg...@googlegroups.com
Thanks.
It works better but there is an issue with hggit extension.

#!python
** Mercurial version (4.3.1+462-07f09995e857). TortoiseHg version
(4.3.1+142-29468f9ee497)
** Command:
** CWD: C:\Program Files\TortoiseHg
** Encoding: cp1252
** Extensions loaded: histedit, rebase, evolve, strip, convert,
mercurial_keyring, hggit
** Python version: 2.7.13 (v2.7.13:a06454b1afa1, Dec 17 2016,
20:53:40) [MSC v.1500 64 bit (AMD64)]
** Windows version: sys.getwindowsversion(major=6, minor=1,
build=7601, platform=2, service_pack='Service Pack 1')
** Processor architecture: x64
** Qt-5.9.1 PyQt-5.9 QScintilla-2.10.1
Traceback (most recent call last):
File "tortoisehg\hgqt\workbench.pyo", line 717, in openRepo
File "tortoisehg\hgqt\repotab.pyo", line 117, in openRepo
File "tortoisehg\hgqt\repotab.pyo", line 383, in _createRepoWidget
File "tortoisehg\hgqt\thgrepo.pyo", line 662, in openRepoAgent
File "tortoisehg\hgqt\thgrepo.pyo", line 62, in repository
File "mercurial\hg.pyo", line 168, in repository
File "mercurial\hg.pyo", line 160, in _peerorrepo
File "hggit\__init__.pyo", line 213, in reposetup
File "hgdemandimport\demandimportpy2.pyo", line 145, in __getattr__
File "hgdemandimport\demandimportpy2.pyo", line 90, in _load
File "hgdemandimport\demandimportpy2.pyo", line 41, in _hgextimport
File "hggit\gitrepo.pyo", line 4, in <module>
ImportError: cannot import name peerrepository

After disabling hggit, no other issue.

André

Steve Borho

unread,
Sep 13, 2017, 2:38:39 PM9/13/17
to thg...@googlegroups.com
thanks, it looks like hg-git does not support the tip of hg default, peerrepository was removed about a month ago

André Sintzoff

unread,
Sep 14, 2017, 2:47:25 AM9/14/17
to thg...@googlegroups.com
It seems I'm no more able to open Terminal from the Repository registry.
No issue to open Explorer.

André

Steve Borho

unread,
Sep 14, 2017, 9:45:45 AM9/14/17
to thg...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages