Qt4/PyQt4 to Qt5/PyQt5 Conversion Notes

[mark.mc...@disneyanimation.com]

Several individuals have asked for more information about Walt Disney Animation Studios' experiences with transitioning from Qt4 to Qt5 (and PyQt4 to PyQt5). Here is more information gleaned from our internal notes. There are two major sections: one on Qt and another on PyQt. Take these notes with a grain of salt. We have not always followed this process.

Qt4 to Qt5 Conversion

Known Issues

• There are some problems with the Qt5 color palette.

Qt4 to Qt4 Update

Strip Qt Header Includes [Optional]

Overview

Update #include statements like this:

#include <QtGui/QWidget>

to this by removing the Qt module directory:

#include <QWidget>

Then add the Qt include modules (or directories) that are needed to a product's Makefile file using -isystem.

fixqt4headers.pl Script

Qt5 comes with a script for stripping Qt4 header includes named fixqt4headers.pl. You may find it useful, because it knows what updates need to be made.

Qt4 Compatibility

The following C++ #if statements can be used to create code that supports both Qt4 and Qt5. Unfortunately this is not so easy with PyQt.

#if QT_VERSION >= QT_VERSION_CHECK(5, 0, 0)

#if QT_VERSION < QT_VERSION_CHECK(5, 0, 0)

 

Qt4 Updates

The following updates can be made now on Qt4 code which are optional in Qt4, but required in Qt5.

• change QString::toAscii() calls to QString::toUtf8()
• change Qt::WFlags to Qt::WindowFlags
• change boolean usages of FALSE and TRUE to standard C++ false and true
• change QWidget::setShown() to QWidget::setVisible()
• change qApp->method() to QApplication::method() where possible
• See information about QListWidget::setItemSelected in the section on Qt5 Updates.
• QAbstractItemModel::reset()
  • Update calls to reset() to calls to beginResetModel() and endResetModel() instead.

Qt4 to Qt5 Conversion

Update Qt Header Includes

Overview

Either strip the Qt Header Includes as described above, or some #include statements like this:

#include <QtGui/QWidget>

will need to be converted this (or something similar):

#include <QtWidgets/QWidget>

Qt5 Updates

• Additional headers may need to be included (most of these are compatible with Qt4), e.g., <QDrag>, <QMimeData>, <QWidgets>,
• QApplication::syncX() is gone and must be replaced with XSync(QX11Info::display(), false)
  • You will need to include <QX11Info> and  <X11/Xlib.h> and link with Qt5X11Extras and X11.
• Styles
• Many of the old Qt4 styles have been removed from Qt5, e.g., Motif, CDE, Cleanlooks and Plastique. These have been replaced with Fusion. We have been replacing QPlastiqueStyle with QProxyStyle in some products.

• QPixmap::grabWindow has been replaced by QScreen::grabWindow.
• The return value of the length method on some classes has been changed from uint to int.
• Replace QListWidget::setItemSelected(item, select) with QListWidgetItem::setSelected(selected), e.g., list->setItemSelected(item, flag) becomes item->setSelected(flag). (The setItemSelected method was actually deprecated in Qt4.)

Qt5 Plugins

• The plugin mechanism in Qt5 has changed. See the documentation on How to Create Qt Plugins for more details.
• In addition, the directory in which Qt plugins are stored has changed from lib64/qt4/plugins to share/qt5/plugins. 

Makefiles

All makefiles will need to be updated by hand. This includes (but is not limited to):

• changing include paths (you will most likely need to add QtWidgets)
• changing Qt library names from Qt<module> to Qt5<module>, e.g., QtGui becomes Qt5Gui
• you will most likely have to link with the new Qt5 widgets library Qt5Widgets
• changing other targets from Moc4 / PyRcc4 / PyRUIC4 / Sip4 to Moc5 / PyRcc5 / PyRUIC5 / Sip5
• you may need to add %Import QtWidgets/QtWidgetsmod.sip to SIP files

Gotchas

The following is a list of issues that people have run into when converting Qt4 code to Qt5 code.

• Qt5, OpenGL, and GLEW
  • QGLWidget is obsolete. Use QOpenGLWidget instead.

  • When using GLEW or GLU with Qt, include files in this order:

    #include <GL/glew.h>

    #include <GL/gl.h>

    #include <GL/glu.h>

    #include <QGLWidget>

    and don't include an entire Qt module like this:

    #include <QtGui>

PyQt4 to PyQt5 Conversion

Documents

Differences Between PyQt4 and PyQt5

QtPy

At this time we are considering porting to QtPy instead of or in addition to Qt5. The QtPy API is similar to that of PyQt5, but under the hood it supports PyQt4.

Known Issues

• QProgressDialog appears automatically.
  • Workaround is to call reset() on the progress dialog instance to prevent it from automatically appearing.
  • https://bugreports.qt.io/browse/QTBUG-47042
• GUIs that use QtWebKitEngine will encounter a segfault on exit
  • [QTBUG-52988] [REG 5.5 -> 5.6] Segfault in QDBusConnectionPrivate::closeConnection -> QObject::disconnect on exit - Qt B…
  • There is no known workaround at the moment, but the linked issue includes patches that we can consider backporting into our Qt5 build.
• If you are using custom X11 (xmodmap) keyboard remapping or layouts then Qt may not properly handle shortcut keys. These bugs may be related.
  • [QTBUG-31527] xmodmap remappings are not respected - Qt Bug Tracker
  • [QTBUG-32322] Qt 5.1 on Linux/X11 ignores xkb keyboard layout - Qt Bug Tracker

The Complete Qt4/PyQt4 to Qt5/PyQt5 Conversion Process

The conversion from Qt4/PyQt4 to Qt5/PyQt5 should be done in three phases as outlined below. Further details about each phase are described in the rest of this document and in Qt4 to Qt5 Conversion.

Preliminaries

• Clean up your old code. Get rid of dead products and code that are no longer being used. This will prevent you from having to do more work than is necessary.

Phase One: Update Qt4 and PyQt4 Code

• If you are maintaining source compatibility, you will need a way to identify the Qt5 environment. At WDAS we use the DISNEY_QT environment variable. If it's undefined, or set to "qt4", then the environment is qt4-centric.  If it's set to "qt5" then the environment is using Qt5.
  
• Update the Qt header includes in your C++ code. See Update Qt Header Includes for details.
• Replace all instances of from PyQt4.Qt<module> import * in your Python code with explicit imports of only the PyQt classes that are being used; or with imports of the PyQt modules, having each class prefixed by the module name. pylint is helpful for finding missing PyQt classes that need to be explicitly imported.

• Update to the new-style PyQt signals and slots (which are optional in PyQt4, but required in PyQt5). See PyQt4 to PyQt4 below for details.

  # Do the PyQt4 update (using pyqt4topyqt5.py --nopyqt5).
  

  # Do the Qt4 to Qt4 conversion. See Qt4 to Qt4 Update for details.

Phase Two: Convert to SIP API 2.0 [Optional]

• The major issue here is that all PyQt4 scripts will have to be updated before the switch to SIP API 2.0 can be thrown. In this regard, this phase is similar to the next phase.
• The major difference between the SIP API 1.0 and 2.0 is the removal of QString and QVariant (in addition to the new-style signals and slots).

Phase Three: Convert to Qt5 and PyQt5

• Convert to Qt5.

   # Do the PyQt5 conversion (usig pyqt4topyqt5.py).

   # Do the Qt5 conversion (manually). See Qt4 to Qt5 Conversion for details.

Final Steps

• Fix up any remaining usages of QString and QStringList (see QString and QStringList below).

  $EDITOR `git grep -l QString`

• Check for any remaining traces of Qt4 and address them.

  git grep -i qt4 

PyQt4 to PyQt4 Update

The pyqt4topyqt5.py script has a --nopyqt5 flag. When using this flag only updates that are compatible with PyQt4 are performed (mainly updating to the new-style signals and slots). Using this flag produces an output file or directory with the name <file>_PyQt4.py or <dir>_PyQt4, and a log file named pyqt4_to_pyqt4.log(unless you use the --nolog flag).

This snippet of shell script has been found to be useful in converting an entire product:

set product=<repo-name>

pyqt4topyqt5.py --nolog --nopyqt5 ${product}

cd ${product}_PyQt4/

find . -type f | cpio -pdmv ../${product}

cd ../${product}/

PyQt4 Gotchas

The following is a list of issues that people have run into when updating PyQt4 code. The vast majority of these issues have to do with the new-style signals and slots. See the PyQt4 Reference Guide New-Style Signals and Slots for more detailed information on the new-style signals and slots. See also the PyQt4 Reference Guide Things to be Aware Of for other gotchas.

• The old-style PyQt signals and slots silently ignored many types of errors in calls to the connect, disconnect, and emit methods. This is not true of the new-style signals and slots which are much stricter. If you find your converted program throwing an exception during one of these calls, it may be due to a preexisting bug and not a problem with the conversion.
• Invalid old-style signals and slots syntax inside of the SIGNAL or SLOT argument string, e.g.,
  • SIGNAL("currentIndexChanged(const QString &") was missing a closing parenthesis and should have been SIGNAL("currentIndexChanged(const QString &)")
  • SIGNAL("updateProgress") was missing arguments and should have been SIGNAL("updateProgress(int)")

• Overloaded signals. This is an example of an overloaded signal problem found:

  . . .

  self.emit(SIGNAL("columnReorderRequested(int,int)"), fromIdx, toIdx)

  self.emit(SIGNAL("columnReorderRequested(const char*,int)"), self._columnBeingDragged, toIdx)

  The automatic conversion script produced this:

  columnReorderRequested = pyqtSignal(int, int)

  . . .

  self.columnReorderRequested.emit(fromIdx, toIdx)

  self.columnReorderRequested.emit(self._columnBeingDragged, toIdx)

  But what was really needed for this overloaded signal was this:

  columnReorderRequested = pyqtSignal([int, int], [str, int])

  . . .

  self.columnReorderRequested[int,int].emit(fromIdx, toIdx)

  self.columnReorderRequested[str,int].emit(self._columnBeingDragged, toIdx)

• String conversion between a signal and a slot. When a signal that is defined as signal = pyqtSignal(str) is connected to a slot that is defined as def slot(self, text), the type of the text argument received by the slot will be QString and not str. This can be avoided by defining a signal as pyqtSignal("const char*") if desired.
• You should also be aware that pyuic4 generates code that uses old-style connections.
• You might have to delete the signal argument type from the converted code. For example, model.itemChanged[QStandardItem].connect(slot) must be changed to model.itemChanged.connect(slot).
• Conversely, you might have to add a signal argument type to the generated pyqtSignal() if no type was specified in the old-style signal string.

• If your signal has the same name as a method of the class in which it is defined, then the signal class member will need to be renamed. For example,

  executeCode = pyqtSignal(str)

  is changed to:

  executeCodeSignal = pyqtSignal(str, name='executeCode') 

PyQt4 to PyQt5 Conversion

The pyqt4topyqt5 product consists of a Python script that automatically converts a PyQt4 script to a PyQt5 script. It handles PyQt module renaming, and conversion from old-style signals and slots to new-style signals and slots among other things. It has a number of limitations, but it is being enhancing it to handle more cases as people come across them.

The conversion script can be run as follows:

pyqt4topyqt5.py <file.py>

pyqt4topyqt5.py <dir>

This will produce an output file or directory with the name <file>_PyQt5.py or <dir>_PyQt5. The script will convert all files with a Python file name extension of .py, and all executable files which contain a Python shebang ("#!/usr/bin/env python" or "#!/usr/bin/python"). If the script is given a single file which does not meet these criteria, it is assumed to be a text file containing a list of file names to convert.

This snippet of shell script has been found to be useful in converting an entire product repo:

set product=<repo-name>

pyqt4topyqt5.py --nolog ${product}

cd ${product}_PyQt5/

find . -type f | cpio -pdmv ../${product}

cd ../${product}/

Note: pyqt4topyqt5.py will create a pyqt4_to_pyqt5.log file every time it is run unless you use the --nolog flag. You may want to clean up these files.

Qt/PyQt Modules

Many of the Qt (and hence PyQt) modules have been split into separate modules.

• QtGui: The QtGui module has been split into QtGui, QtWidgets, and QtPrintSupport.
• QtOpenGL: Only the QGLContext, QGLFormat and QGLWidget classes are supported by PyQt5.
• QtWebKit: The QtWebKit module has been split into QtWebKit and QtWebKitWidgets.
• pyqtconfig: PyQt4’s pyqtconfig module is not supported in PyQt5.

Note: pyqt4topyqt5.py handles most these changes automatically. However, if a Python script imports all symbols from a Qt module (e.g., from PyQt4.QtWebKit import *) there is a good chance that something will be missed. 

QString and QStringList

The QString and QStringList classes are no longer supported in PyQt5. PyQt5 uses Python's native string types (unicode in Python 2). All QString instances are automatically converted to and from Python unicode strings in PyQt5. Instances of QStringList are converted to and from Python lists of unicode strings. The following is a list of QString method conversions that need to be done by hand.

• Calls to QString.toAscii() need to be removed or replaced with str().
• Calls to other QString methods need to be replaced with their Python string equivalents:
  • string.trimmed() --> string.strip()
  • string.startsWith() --> string.startswith()
  • string.endsWith() --> string.endswith()
  • string.isNull() --> not string
  • string.length() --> len(string)
• Calls to QString.arg() need to be replaced with Python string formatting.
• If the conversion script runs across instances of QString that it does not know how to convert, the following will be added to the top of the Python file. You may want to remove this line and convert the references to QString by hand yourself.
  • QString = unicode
• The conversion script does not handle QStringLists. Those must be dealt with manually.

QVariant

In PyQt5 the implementation of QVariant is different to those of PyQt4. By default the behavior is the same as PyQt4’s v2 API. PyQt4’s v2 API does not expose the QVariant class to Python and automatically converts a QVariant to the actual value. See Support for QVariant.

Calls to QVariant.toString() need to be removed or replaced with unicode().

QThread

If you subclassed QThread and reimplemented the run() method as a workaround to a problem in PyQt4.6 (or prior versions), remove it from the subclass. The workaround is no longer needed. See Background thread with QThread in PyQt.

QSettings

If you use QSettings.setValue with a boolean value

boolValue = False

settings.setValue("foo", boolValue)

AND it writes out to a file, be aware that when you read the value

flag = settings.value("foo")

the value (flag) will be a unicode string, not a boolean value.

Other Differences

• QStyle.standardPixmap is deprecated, and QStyle.standardIcon should be used instead.

Gotchas

The following is a list of issues that people have run into when converting PyQt4 code to PyQt5 code. See also the PyQt5 Reference Guide Things to be Aware Of for more PyQt5 gotchas.

• When converting from PyQt old-style signals and slots to the new style, if any of the signals take arguments of a Qt type, then that Qt type must be imported or the places where that type occurs within signal/slot square brackets must be prefixed with QtCore., QtWidgets., or QtGui..
• There is a Qt config file in ~/.config, called Trolltech.conf.  A few people have noticed that this file can become very large (greater than 10 MB) when you go back and forth between Qt4 and Qt5, and Qt performance can degrade as the file becomes larger.  If you experience UI performance issues, try moving this file aside.

Limitations

In addition to the limitations mentioned above, the following have also been uncovered:

• PyQt4 code that is inside comments or strings will not be converted. This also applies to code within files of other types like C++ or MEL.

[François Beaune]

Very much appreciated, thanks!

Franz

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-discuss+unsub...@googlegroups.com.
To post to this group, send email to vfx-platform-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/ad661c61-1eef-49cb-933c-afb47869b186%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[innerh...@gmail.com]

Hi Mark,

Thanks for this useful info.

I notice that the above link to the abstraction library is QtPy (https://github.com/spyder-ide/qtpy), and not Qt.py (https://github.com/mottosso/Qt.py), which has been referenced in previous threads and claims to be "born in the film and visual effects industry".

I've tried both implementation and they both seem to do the job. Is there a preference? I'd prefer to opt for the one which has most take up in vfx, as we're likely to run into similar problems.

Thanks,

Will

[David Aguilar]

I am using QtPy (spyder) primarily because it was the first one I
found that was easy to try. I ported git-cola to it, and most of the
tools at WDAS that I touch either have or will be moving to it.

My original motivation was not WDAS studio-centric -- spyder-ide/qtpy
was already packaged in Debian so it was the easiest one for me to try
-- but it is the one that we will use because we've already packaged
it and I did go into the port with the assumption that whatever I
learned would carry over to porting the studio's tools. Porting
git-cola shook out most of the issues and helped me find the subset of
functionality that is common to all of them. Discovering these
details was harder than getting the shim in place.

From looking at the implementation, it should be fine to mix and match
them. They're both import wrappers.

One small difference in the implementations is that the spyder/QtPy
modules try very hard to present the new-style view to users of the
modules. They do this by providing modules that selectively import
things into their namespace. In Qt.py it looks like the remapping is
done by doing e.g. "QtWidgest = QtGui", which means that some modules
may contain, depending on the backend, symbols that are not present in
other backends, and vice versa. Changing that would require moving
away from the single-file implementation, so it may not align with
Qt.py's goals. Trying to make that consistent is helpful since I tend
to develop in one backend and do testing in another, but it's a pretty
minor issue.

spyder/QtPy is over a year older (Feb 2015 vs. May 2016) so that may
explain why it was in Debian at the time. It's a spyder-ide
dependency so fedora and friends are already packaging it:
http://koji.fedoraproject.org/koji/packageinfo?packageID=23135


That said, please don't take our usage of spyder/QtPy as an
anti-endorsement of Qt.py -- I just wanted to clarify that detail.
Sorry for the confusion!
--
David

[Marcus Ottosson]

Hi, co-author of Qt.py here!

Out of the similar projects, QtPy is the most similar.

The main difference, and the aspect we focus the most on, is to not break the original binding on import, such that running software written without Qt.py does not break. QtPy doesn’t have this requirement, as their software runs alone within a single Python interpreter process, but for us running tens if not hundreds of them in Maya et. al. problems amongst minor differences multiply like bunnies.

None of the other projects take this into consideration and the biggest culprit we’ve found so far is monkey-patching classes which causes subtle and difficult to debug problems.

To combat this, Qt.py is run through extensive testing on each push, for features, presence of members relative its original bindings, known caveats and workarounds and hand-written examples.

The second biggest difference is that Qt.py uses PySide2 as a reference binding. Which means any code written for PySide2 will work with Qt.py, which enables the use of PySide2’s ui-compiler. QtPy doesn’t have a reference binding but rather mixes arbitrarily based on preference and what works for their parent qt-spider project.

Not to bash QtPy, I think it’s important to point out that the projects are actually really similar and that they’ve done nothing wrong - we just have different requirements. The one showstopper that I would think about before you make your decision about which to choose is whether you are willing to take the risk of making changes to the original binding and potentially break currently working software.

And as for preference, we’re trying to keep tabs on who’s using it so as others can see whether the userbase relates to them or not. Speaking of which, if you know of any studio not in this list using Qt.py, let me know!

Best,
Marcus

​

--
You received this message because you are subscribed to the Google Groups "vfx-platform-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vfx-platform-discuss+unsub...@googlegroups.com.
To post to this group, send email to vfx-platform-discuss@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vfx-platform-discuss/CAGd_kMAO9RcekU3Wxbd24C_24GsaRBu%3DRG4uJQiwqRUK1MrFJA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Marcus Ottosson
konstr...@gmail.com