[PATCH 1 of 6] tests: close all items that were opened so Windows can cleanup

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>
# Date 1735942763 18000
# Fri Jan 03 17:19:23 2025 -0500
# Branch stable
# Node ID de6a993bd50f584211cc6a9fa5a620bd942a9972
# Parent 0ff1072c3279ff18a1a6481e3287e711f273fc01
# EXP-Topic windows-test-fixes
tests: close all items that were opened so Windows can cleanup

I suspect it was the change in `wconfig_test.py`, but at the end of a test run
on Windows, there was a large stacktrace trying to delete the temp directory:

<snip>
File "c:\Users\Matt\projects\mercurial\thg\tests\pytesthgenv.py", line 131, in pytest_sessionfinish
shutil.rmtree(self.tmpdir)
File "C:\hgdev\python39-x64\lib\shutil.py", line 759, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\hgdev\python39-x64\lib\shutil.py", line 624, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\hgdev\python39-x64\lib\shutil.py", line 629, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\hgdev\python39-x64\lib\shutil.py", line 627, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\Matt\\AppData\\Local\\Temp\\thgtests.azbole4u\\wconfig_testc231xs2q\\tmp6xq5o86w'

diff --git a/tests/qt_cmdagent_test.py b/tests/qt_cmdagent_test.py
--- a/tests/qt_cmdagent_test.py
+++ b/tests/qt_cmdagent_test.py
@@ -339,9 +339,13 @@
buf = QBuffer()
buf.setData(b'foo')
buf.open(QIODevice.OpenModeFlag.ReadOnly)
- sess = self.agent.runCommand(['echoback'])
- sess.setInputDevice(buf)
- self._check_runcommand(sess, 'foo')
+
+ try:
+ sess = self.agent.runCommand(['echoback'])
+ sess.setInputDevice(buf)
+ self._check_runcommand(sess, 'foo')
+ finally:
+ buf.close()

def test_data_input_unset(self):
sess = self.agent.runCommand(['echoback'])
diff --git a/tests/wconfig_test.py b/tests/wconfig_test.py
--- a/tests/wconfig_test.py
+++ b/tests/wconfig_test.py
@@ -447,4 +447,5 @@
c = newwconfig({b'foo.bar': b'baz'})
fname = writetempfile(b'')
wconfig.writefile(c, fname)
- assert open(fname, 'rb').read().rstrip() == b'[foo]\nbar = baz'
+ with open(fname, 'rb') as fp:
+ assert fp.read().rstrip() == b'[foo]\nbar = baz'

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>

# Date 1735943428 18000
# Fri Jan 03 17:30:28 2025 -0500
# Branch stable
# Node ID c9d03ead1019d78de14518b6bfaa55c9d1058b5f
# Parent de6a993bd50f584211cc6a9fa5a620bd942a9972
# EXP-Topic windows-test-fixes
wconfig_test: expect the configuration to be written with native EOL

This was failing on Windows, but I expect it's more reasonable to use native
endings there because of notepad prior to Windows 11.

diff --git a/tests/wconfig_test.py b/tests/wconfig_test.py
--- a/tests/wconfig_test.py
+++ b/tests/wconfig_test.py

@@ -447,5 +447,7 @@

c = newwconfig({b'foo.bar': b'baz'})
fname = writetempfile(b'')
wconfig.writefile(c, fname)

+
+ expected = b'[foo]%sbar = baz' % os.linesep.encode('ascii')

with open(fname, 'rb') as fp:

- assert fp.read().rstrip() == b'[foo]\nbar = baz'
+ assert fp.read().rstrip() == expected

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>

# Date 1735944376 18000
# Fri Jan 03 17:46:16 2025 -0500
# Branch stable
# Node ID b143524983fec3327019557a5e336e96c015fcf3
# Parent c9d03ead1019d78de14518b6bfaa55c9d1058b5f
# EXP-Topic windows-test-fixes
tests: fix a NameError in qt_manifestmodel_test.py on Windows and macOS

I assume this is what it was flagging as a NameError- there was no stacktrace.

ERROR tests/qt_manifestmodel_test.py::ManifestModelEucjpTest::test_data - NameError: name 'SkipTest' is not defined
ERROR tests/qt_manifestmodel_test.py::ManifestModelEucjpTest::test_fileicon_path_concat - NameError: name 'SkipTest' is not defined
ERROR tests/qt_manifestmodel_test.py::ManifestModelEucjpTest::test_indexfrompath - NameError: name 'SkipTest' is not defined
ERROR tests/qt_manifestmodel_test.py::ManifestModelEucjpTest::test_pathfromindex - NameError: name 'SkipTest' is not defined

diff --git a/tests/qt_manifestmodel_test.py b/tests/qt_manifestmodel_test.py
--- a/tests/qt_manifestmodel_test.py
+++ b/tests/qt_manifestmodel_test.py
@@ -684,7 +684,7 @@
def setUpClass(cls):
# TODO: make this compatible with binary-unsafe filesystem
if os.name != 'posix' or sys.platform == 'darwin':
- raise SkipTest
+ raise unittest.SkipTest
cls.encodingpatch = helpers.patchencoding('euc-jp')

# include non-ascii char in repo path to test concatenation

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>

# Date 1736546216 18000
# Fri Jan 10 16:56:56 2025 -0500
# Branch stable
# Node ID 43c4af2c5f542a8a9e547c0b735ce26b91af03b2
# Parent b143524983fec3327019557a5e336e96c015fcf3
# EXP-Topic windows-test-fixes
tests: skip QTextCodec dependent tests with Qt6 (fixes #6010)

This class was removed in Qt6, and it's not obvious how to replace the
functionality that went missing. If someone cares, they can resurrect this.

diff --git a/tests/helpers.py b/tests/helpers.py
--- a/tests/helpers.py
+++ b/tests/helpers.py
@@ -8,9 +8,14 @@
import time
from collections import defaultdict

-from tortoisehg.hgqt.qtcore import (
- QTextCodec,
-)
+try:
+ from tortoisehg.hgqt.qtcore import (
+ QTextCodec,
+ )
+except ImportError:
+ # QStringConverter is supposed to be the replacement on Qt6, but is missing
+ # a lot of stuff.
+ QTextCodec = None

from mercurial import (
dispatch,
diff --git a/tests/hglib_encoding_test.py b/tests/hglib_encoding_test.py
--- a/tests/hglib_encoding_test.py
+++ b/tests/hglib_encoding_test.py
@@ -8,6 +8,7 @@

JAPANESE_KANA_I = u'\u30a4' # Japanese katakana "i"

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('utf-8')
def test_none():
"""None shouldn't be touched"""
@@ -16,59 +17,71 @@
assert f(None) is None


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('utf-8')
def test_fromunicode():
assert hglib.fromunicode(JAPANESE_KANA_I) == JAPANESE_KANA_I.encode('utf-8')

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii', 'utf-8')
def test_fromunicode_fallback():
assert hglib.fromunicode(JAPANESE_KANA_I) == JAPANESE_KANA_I.encode('utf-8')

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_fromunicode_replace():
assert hglib.fromunicode(JAPANESE_KANA_I, errors='replace') == b'?'

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_fromunicode_strict():
with pytest.raises(UnicodeEncodeError):
hglib.fromunicode(JAPANESE_KANA_I)


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('euc-jp')
def test_fromutf():
assert (hglib.fromutf(JAPANESE_KANA_I.encode('utf-8'))
== JAPANESE_KANA_I.encode('euc-jp'))

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii', 'euc-jp')
def test_fromutf_fallback():
assert (hglib.fromutf(JAPANESE_KANA_I.encode('utf-8'))
== JAPANESE_KANA_I.encode('euc-jp'))

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_fromutf_replace():
assert hglib.fromutf(JAPANESE_KANA_I.encode('utf-8')) == b'?'


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('euc-jp')
def test_tounicode():
assert hglib.tounicode(JAPANESE_KANA_I.encode('euc-jp')) == JAPANESE_KANA_I

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii', 'euc-jp')
def test_tounicode_fallback():
assert hglib.tounicode(JAPANESE_KANA_I.encode('euc-jp')) == JAPANESE_KANA_I


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('euc-jp')
def test_toutf():
assert (hglib.toutf(JAPANESE_KANA_I.encode('euc-jp'))
== JAPANESE_KANA_I.encode('utf-8'))

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii', 'euc-jp')
def test_toutf_fallback():
assert (hglib.toutf(JAPANESE_KANA_I.encode('euc-jp'))
== JAPANESE_KANA_I.encode('utf-8'))


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('gbk')
def test_gbk_roundtrip():
# gbk byte sequence can also be interpreted as utf-8 (issue #3299)
@@ -77,12 +90,14 @@
assert hglib.tounicode(l) == MOKU


+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_lossless_unicode_replaced():
l = hglib.fromunicode(JAPANESE_KANA_I, 'replace')
assert l == b'?'
assert hglib.tounicode(l) == JAPANESE_KANA_I

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('euc-jp')
def test_lossless_unicode_double_mapped():
YEN = u'\u00a5' # "yen" and "back-slash" are mapped to the same code
@@ -90,6 +105,7 @@
assert l == b'\\'
assert hglib.tounicode(l) == YEN

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_lossless_utf_replaced():
u = JAPANESE_KANA_I.encode('utf-8')
@@ -97,6 +113,7 @@
assert l == b'?'
assert hglib.toutf(l) == u

+...@pytest.mark.skipif(helpers.QTextCodec is None, reason="Qt6 not supported")
@helpers.with_encoding('ascii')
def test_lossless_utf_cannot_roundtrip():
u = JAPANESE_KANA_I.encode('cp932') # bad encoding

diff --git a/tests/qt_manifestmodel_test.py b/tests/qt_manifestmodel_test.py
--- a/tests/qt_manifestmodel_test.py
+++ b/tests/qt_manifestmodel_test.py

@@ -685,6 +685,10 @@

# TODO: make this compatible with binary-unsafe filesystem
if os.name != 'posix' or sys.platform == 'darwin':

raise unittest.SkipTest
+ # TODO: figure out the replacement for QTextCodec on Qt6
+ if helpers.QTextCodec is None:
+ raise unittest.SkipTest
+

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>

# Date 1736554101 18000
# Fri Jan 10 19:08:21 2025 -0500
# Branch stable
# Node ID 6bd7bd92060946d818f524d8b2d2c25b6a7196fc
# Parent 43c4af2c5f542a8a9e547c0b735ce26b91af03b2
# EXP-Topic windows-test-fixes
tests: fix a cast of QByteArray to bytes

This was flagged by PyCharm, but apparently harmless. I confirmed that
`readAll()` returns `PyQt6.QtCore.QByteArray`, and doesn't have `__bytes__()`
in the type stub for that class. Still, the previous code ran (when the errors
leading up to it were disabled). We have a library function for this, so use it
to suppress the warning.

diff --git a/tests/qt_cmdagent_test.py b/tests/qt_cmdagent_test.py
--- a/tests/qt_cmdagent_test.py
+++ b/tests/qt_cmdagent_test.py

@@ -19,7 +19,10 @@
pyqtSlot,
)

-from tortoisehg.hgqt import cmdcore
+from tortoisehg.hgqt import (
+ cmdcore,
+ qtlib,
+)
from tortoisehg.util import hglib

import helpers
@@ -145,7 +148,8 @@
self.assertEqual(self._strhgpath() + '\n', sess.readLine())
self.assertEqual(b'5', sess.read(1))
self.assertEqual(b'3', sess.peek(1))
- self.assertEqual(b'3245c60e682\n', bytes(sess.readAll()))
+ self.assertEqual(b'3245c60e682\n',
+ qtlib.qbytearray_or_bytes_to_bytes(sess.readAll()))

def _check_runcommand(self, sess, expectedout, expectedcode=0):
self.assertFalse(sess.isFinished())

[Matt Harbison]

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>

# Date 1736547638 18000
# Fri Jan 10 17:20:38 2025 -0500
# Branch stable
# Node ID 36cde4eb7541a7379d087659ec9029a711249758
# Parent 6bd7bd92060946d818f524d8b2d2c25b6a7196fc
# EXP-Topic windows-test-fixes
xxx-tests: hacking on CmdSession capture failure with Qt6

This is definitely not the right way to do it, but there are multiple issues
here in Qt6.

- sess.readLine() returns None, and stderr says "QIODevice::readLine (QBuffer):
Called with maxSize < 2". The `maxlen` default arg for readLine() is 0, and
I suspect that it's tripping over this[1]. I suspect this method was used for
efficiency (instead of using readLineData(), which always return bytes), so is
there a better way than either passing a stupid large number, or .peek() + .read(1)
in a loop?

- Once that's skipped, the assertion fails because `self._strhgpath()` is str,
and the line read back is bytes. CmdSession.readLine() is typed to return
bytes (but the QDevice.readLine() it delegates to says it can be bytes (if maxlen
is passed), or QByteArray (if not passed, which is what I'd expect a 0 arg to
decay to). I've confirmed that it's also returning bytes on Qt5. 581f5bcc722c
doesn't say why it was converted to str, but the other places were using it
as a command arg, which should be str. Maybe this was accidentally swept up
in the change. I have no clue how .assertEqual with Qt5 isn't failing for the
different types though. Both venvs are using pytest 8.3.4 with Python 3.9.21.

[1] https://codebrowser.dev/qt6/qtbase/src/corelib/io/qiodevice.cpp.html#1341

diff --git a/tests/qt_cmdagent_test.py b/tests/qt_cmdagent_test.py
--- a/tests/qt_cmdagent_test.py
+++ b/tests/qt_cmdagent_test.py

@@ -145,7 +145,7 @@
self.assertTrue('no such file' in sess.warningString()) # not captured
self.assertTrue(readyRead.called)
self.assertTrue(sess.canReadLine())
- self.assertEqual(self._strhgpath() + '\n', sess.readLine())
+ self.assertEqual(self.hg.path + b'\n', sess.readLine(5000))

self.assertEqual(b'5', sess.read(1))
self.assertEqual(b'3', sess.peek(1))

self.assertEqual(b'3245c60e682\n',

[Matt Harbison]

On Friday, January 10, 2025 at 7:47:32 PM UTC-5 Matt Harbison wrote:

# HG changeset patch
# User Matt Harbison <matt_h...@yahoo.com>
# Date 1736547638 18000
# Fri Jan 10 17:20:38 2025 -0500
# Branch stable
# Node ID 36cde4eb7541a7379d087659ec9029a711249758
# Parent 6bd7bd92060946d818f524d8b2d2c25b6a7196fc
# EXP-Topic windows-test-fixes
xxx-tests: hacking on CmdSession capture failure with Qt6

Obviously this one isn't a serious attempt, and the real fix is likely to be some change in the CmdSession.readLine() implementation.  I just don't understand what they changed and why, and if there are other hidden effects.  But these changes give us a clean run with PyQt5 and PyQt6 on Linux.  (I sorta gave up on Windows because there are a bunch of other platform differences like using '\\' vs '/' in one of the models.)

[Yuya Nishihara]

On Fri, 10 Jan 2025 19:46:43 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_h...@yahoo.com>
> # Date 1735942763 18000
> # Fri Jan 03 17:19:23 2025 -0500
> # Branch stable
> # Node ID de6a993bd50f584211cc6a9fa5a620bd942a9972
> # Parent 0ff1072c3279ff18a1a6481e3287e711f273fc01
> # EXP-Topic windows-test-fixes
> tests: close all items that were opened so Windows can cleanup

Queued, thanks.

> buf = QBuffer()
> buf.setData(b'foo')
> buf.open(QIODevice.OpenModeFlag.ReadOnly)
> - sess = self.agent.runCommand(['echoback'])
> - sess.setInputDevice(buf)
> - self._check_runcommand(sess, 'foo')
> +
> + try:
> + sess = self.agent.runCommand(['echoback'])
> + sess.setInputDevice(buf)
> + self._check_runcommand(sess, 'foo')
> + finally:
> + buf.close()

Just fyi, QBuffer is in-memory buffer, so .close() wouldn't matter here.
It's nicer to do, though.

[Yuya Nishihara]

Hmm, things might be broken in PyQt? This readLine() is supposed to do
call readLine(maxSize = 0). We might have to omit the default explicitly.
https://codebrowser.dev/qt6/qtbase/src/corelib/io/qiodevice.cpp.html#_ZN9QIODevice8readLineEx
https://doc.qt.io/qt-6/qiodevice.html#readLine-1