[PATCH 0 of 6] Fix compile- and run-time issues for CentOS

[Mathias De Mare]

Hello,

these patches address either issues when building for CentOS,
or runtime problems (for example passing additional arguments).

Greetings,
Mathias

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>
# Date 1625818129 -7200
# Fri Jul 09 10:08:49 2021 +0200
# Branch stable
# Node ID 8af452b8cb648ed2f999f2c12e00435ef5e77a67
# Parent 2d510130c6260ff378a629a0b448659a4358be8b
tortoisehg.spec: allow command-line arguments to be passed along

This fixes a bug where 'thg log <subdirectory>' no longer worked on CentOS
(and neither did any other additional thg arguments).

diff --git a/contrib/tortoisehg.spec b/contrib/tortoisehg.spec
--- a/contrib/tortoisehg.spec
+++ b/contrib/tortoisehg.spec
@@ -81,7 +81,7 @@ export PYTHONPATH=%{_prefix}/local/lib/t
export PYTHONPATH=%{_prefix}/local/lib/thg-env/lib64/python3.6/site-packages/:\$PYTHONPATH
export PYTHONPATH=%{_prefix}/local/lib/thg-env/usr/lib/python3.6/site-packages/:\$PYTHONPATH
export PATH=%{_prefix}/local/lib/thg-env/bin:\$PATH
-%{_prefix}/local/lib/thg-env/usr/bin/thg
+%{_prefix}/local/lib/thg-env/usr/bin/thg \$@
EOF
chmod +x %{buildroot}%{_prefix}/local/bin/thg

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>

# Date 1625828440 -7200
# Fri Jul 09 13:00:40 2021 +0200
# Branch stable
# Node ID d43fa777c2295edc0c70aca6e0e3dd80233acd8e
# Parent 83ee7408e65e91b00411e8f1b12890a4dd3b6785
contrib: enable rust optimizations in Mercurial shipped with CentOS

This avoids Tortoisehg not properly starting because
of a mismatch between the installed Mercurial and Tortoisehg versions.

diff --git a/contrib/docker/centos7 b/contrib/docker/centos7
--- a/contrib/docker/centos7
+++ b/contrib/docker/centos7
@@ -42,6 +42,9 @@ RUN yum install -y git openssl-devel &&
git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake --build . --target install

+# Install rust for rust-based optimizations
+RUN yum install -y cargo
+
# Configure UTF-8 to avoid python3 issues
ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8
diff --git a/contrib/docker/centos8 b/contrib/docker/centos8
--- a/contrib/docker/centos8
+++ b/contrib/docker/centos8
@@ -44,5 +44,8 @@ RUN yum install -y git openssl-devel &&
git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake --build . --target install

+# Install rust for rust-based optimizations
+RUN yum install -y cargo
+
# wish is needed for Mercurial
RUN yum install tk -y

diff --git a/contrib/tortoisehg.spec b/contrib/tortoisehg.spec
--- a/contrib/tortoisehg.spec
+++ b/contrib/tortoisehg.spec

@@ -56,6 +56,8 @@ rm -rf $RPM_BUILD_ROOT
echo $RPM_BUILD_ROOT

pip3 install -r %{requirements_txt} --prefix=%{buildroot}%{_prefix}/local/lib/thg-env --ignore-installed
+# Mercurial binaries for CentOS are installed using rust support, so let's do the same for Mercurial shipped with Tortoisehg for CentOS
+pip3 install mercurial --prefix=%{buildroot}%{_prefix}/local/lib/thg-env --ignore-installed --global-option="--rust"
export PYTHONPATH=%{buildroot}%{_prefix}/local/lib/thg-env/lib/python3.6/site-packages:%{buildroot}%{_prefix}/local/lib/thg-env/lib64/python3.6/site-packages

python3 setup.py install -O1 --skip-build --root %{buildroot}%{_prefix}/local/lib/thg-env

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>

# Date 1625825035 -7200
# Fri Jul 09 12:03:55 2021 +0200
# Branch stable
# Node ID 83ee7408e65e91b00411e8f1b12890a4dd3b6785
# Parent 3693fa48c810f5fba808c076148d320d67ee128b
contrib centos8: add requirements for libgit2

cmake on CentOS 8.3 has an issue during the libgit2 build:
cmake: symbol lookup error: cmake: undefined symbol: archive_write_add_filter_zstd

There is not CentOS 8.4 docker image yet, so switch to rockylinux instead
to continue being able to build the rpm.

diff --git a/contrib/docker/centos8 b/contrib/docker/centos8
--- a/contrib/docker/centos8
+++ b/contrib/docker/centos8

@@ -1,4 +1,4 @@
-FROM centos:centos8
+FROM rockylinux/rockylinux:8.4
RUN yum install -y \
gcc \
gettext \
@@ -38,5 +38,11 @@ RUN yum install -y \
# Install Mercurial (so it can be called in our scripts)
RUN pip3 install mercurial

+# Install libgit2 to allow usage of pygit2
+RUN yum install -y git openssl-devel && \
+ mkdir /libgit2 && \
+ git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
+ mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake --build . --target install

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>

# Date 1625829792 -7200
# Fri Jul 09 13:23:12 2021 +0200
# Branch stable
# Node ID 5da9d3750dfad628a0d87794c84ac193c98aefc7
# Parent d43fa777c2295edc0c70aca6e0e3dd80233acd8e
hgqt: pass along bytes-based object to Mercurial pprint function

This avoids a crash when passing '--profile' because the pprint
function assumes bytes instead of str.

diff --git a/tortoisehg/hgqt/run.py b/tortoisehg/hgqt/run.py
--- a/tortoisehg/hgqt/run.py
+++ b/tortoisehg/hgqt/run.py
@@ -457,7 +457,7 @@ def _runcommand(ui, options, cmd, cmdfun
path = ui.expandpath(output)
ostream = open(path, 'wb')
else:
- ostream = sys.stderr
+ ostream = sys.stderr.buffer

try:
from mercurial import lsprof

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>

# Date 1625825022 -7200
# Fri Jul 09 12:03:42 2021 +0200
# Branch stable
# Node ID 3693fa48c810f5fba808c076148d320d67ee128b
# Parent 75ac0c2fc5c779a0a02880abbb2e0d23cdd3d779
contrib centos7: add requirements for libgit2

diff --git a/contrib/docker/centos7 b/contrib/docker/centos7
--- a/contrib/docker/centos7
+++ b/contrib/docker/centos7

@@ -30,6 +30,18 @@ RUN yum install -y \

# Install Mercurial (so it can be called in our scripts)

RUN pip install mercurial

+# Install recent cmake to allow libgit2 installation
+RUN yum install -y wget && wget https://github.com/Kitware/CMake/releases/download/v3.20.5/cmake-3.20.5-linux-x86_64.sh && \
+ chmod +x ./cmake-3.20.5-linux-x86_64.sh && ./cmake-3.20.5-linux-x86_64.sh --skip-license --include-subdir --prefix=/ && \
+ ln -s /cmake-3.20.5-linux-x86_64/bin/cmake /usr/bin/cmake
+

+# Install libgit2 to allow usage of pygit2

+#RUN yum install -y libgit2 libgit2-devel

+RUN yum install -y git openssl-devel && \
+ mkdir /libgit2 && \
+ git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
+ mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake --build . --target install
+

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>
# Date 1625823240 -7200

# Fri Jul 09 11:34:00 2021 +0200
# Branch stable
# Node ID 75ac0c2fc5c779a0a02880abbb2e0d23cdd3d779
# Parent 8af452b8cb648ed2f999f2c12e00435ef5e77a67
doc: use python3-compatible syntax

diff --git a/doc/source/conf.py b/doc/source/conf.py
--- a/doc/source/conf.py
+++ b/doc/source/conf.py
@@ -168,8 +168,8 @@ htmlhelp_basename = 'TortoiseHg'
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, document class [howto/manual]).
latex_documents = [
- ('index', 'TortoiseHg.tex', ur'TortoiseHg Documentation',
- ur'Steve Borho and others', 'manual'),
+ ('index', 'TortoiseHg.tex', u'TortoiseHg Documentation',
+ u'Steve Borho and others', 'manual'),
]

# The name of an image file (relative to this directory) to place at the top of

[Matt Harbison]

I don't know anything about this file, but does $@ need to be quoted to avoid breaking arguments with spaces?

[Yuya Nishihara]

Just curious. Maybe this would also apply to pip, but is it common to run
the whole build process as root in docker?

[Yuya Nishihara]

Yes. Fixed it and queued 1-5 for stable, thanks.

[Yuya Nishihara]

procutil.stderr can be used instead. We still need to support Python 2.

[Mathias De Mare]

# HG changeset patch
# User Mathias De Mare <mathias...@nokia.com>
# Date 1625829792 -7200
# Fri Jul 09 13:23:12 2021 +0200
# Branch stable

# Node ID e305fb3f7d30684165cdb1742d66367717aa4817
# Parent 08a745d3e5999ffd41af1ed3c3f4edb2ba9aea82

hgqt: pass along bytes-based object to Mercurial pprint function

This avoids a crash when passing '--profile' because the pprint
function assumes bytes instead of str.

diff --git a/tortoisehg/hgqt/run.py b/tortoisehg/hgqt/run.py
--- a/tortoisehg/hgqt/run.py
+++ b/tortoisehg/hgqt/run.py
@@ -457,7 +457,7 @@ def _runcommand(ui, options, cmd, cmdfun
path = ui.expandpath(output)
ostream = open(path, 'wb')
else:
- ostream = sys.stderr

+ ostream = procutil.stderr

[De Mare, Mathias (Nokia - BE/Antwerp)]

> -----Original Message-----
> From: Yuya Nishihara <you...@gmail.com> On Behalf Of Yuya Nishihara
> Sent: Saturday, July 10, 2021 1:16 PM
> To: thg...@googlegroups.com
> Cc: De Mare, Mathias (Nokia - BE/Antwerp) <mathias...@nokia.com>
> Subject: Re: [PATCH 3 of 6] contrib centos7: add requirements for libgit2
>
> On Fri, 09 Jul 2021 14:26:46 +0200, Mathias De Mare wrote:
> > # HG changeset patch
> > # User Mathias De Mare <mathias...@nokia.com> # Date
> 1625825022
> > -7200
> > # Fri Jul 09 12:03:42 2021 +0200
> > # Branch stable
> > # Node ID 3693fa48c810f5fba808c076148d320d67ee128b
> > # Parent 75ac0c2fc5c779a0a02880abbb2e0d23cdd3d779
> > contrib centos7: add requirements for libgit2
> >
> > diff --git a/contrib/docker/centos7 b/contrib/docker/centos7
> > --- a/contrib/docker/centos7
> > +++ b/contrib/docker/centos7
> > @@ -30,6 +30,18 @@ RUN yum install -y \ # Install Mercurial (so it
> > can be called in our scripts) RUN pip install mercurial
> >

> > +# Install recent cmake to allow libgit2 installation RUN yum install
> > +-y wget && wget
> https://github.com/Kitware/CMake/releases/download/v3.20.5/cmake-

> 3.20.5-linux-x86_64.sh && \
> > + chmod +x ./cmake-3.20.5-linux-x86_64.sh && ./cmake-3.20.5-linux-
> x86_64.sh --skip-license --include-subdir --prefix=/ && \
> > + ln -s /cmake-3.20.5-linux-x86_64/bin/cmake /usr/bin/cmake
> > +

> > +# Install libgit2 to allow usage of pygit2 #RUN yum install -y
> > +libgit2 libgit2-devel RUN yum install -y git openssl-devel && \

> > + mkdir /libgit2 && \
> > + git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
> > + mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake

> > +--build . --target install

>
> Just curious. Maybe this would also apply to pip, but is it common to run the
> whole build process as root in docker?

As far as I know, it is. I've seen a lot of discussion about running containers as non-root, but not much about building containers as non-root (probably because it's very practical to build a container as root).

I guess in the above case, we could alternatively create a user, switch to it and do the cmake and libgit2 install using that separate user. It sounds like a more involved approach though, and I'm not sure it adds much value.

This question does make me realize that for cmake, we retrieve the file without any checksumming, I'll submit a patch for that.

Greetings,
Mathias

[Yuya Nishihara]

On Mon, 12 Jul 2021 08:00:46 +0200, Mathias De Mare wrote:
> # HG changeset patch
> # User Mathias De Mare <mathias...@nokia.com>
> # Date 1625829792 -7200
> # Fri Jul 09 13:23:12 2021 +0200
> # Branch stable
> # Node ID e305fb3f7d30684165cdb1742d66367717aa4817
> # Parent 08a745d3e5999ffd41af1ed3c3f4edb2ba9aea82
> hgqt: pass along bytes-based object to Mercurial pprint function

Queued for stable, thanks.

[Yuya Nishihara]

On Mon, 12 Jul 2021 06:12:50 +0000, De Mare, Mathias (Nokia - BE/Antwerp) wrote:
> > > +# Install libgit2 to allow usage of pygit2 #RUN yum install -y
> > > +libgit2 libgit2-devel RUN yum install -y git openssl-devel && \
> > > + mkdir /libgit2 && \
> > > + git clone -b v1.1.1 https://github.com/libgit2/libgit2.git /libgit2 && \
> > > + mkdir /libgit2/build && cd /libgit2/build && cmake .. && cmake
> > > +--build . --target install
> >
> > Just curious. Maybe this would also apply to pip, but is it common to run the
> > whole build process as root in docker?
> As far as I know, it is. I've seen a lot of discussion about running containers as non-root, but not much about building containers as non-root (probably because it's very practical to build a container as root).

Yep, building container image as root (of the container environment) makes
sense. Maybe we could move libgit2 installation out of the "build a container"
step, but I don't know if that will improve the situation.

Anyway, thanks for the comment.