Dropping ibus-pinyin (or salvaging libpyzy)?

[Boyuan Yang]

[Please use debian-in...@lists.debian.org as main discussion list]

Dear all,

TL;DR: Let's drop package ibus-pinyin from Debian since it implicitly (build-
)depends on the unmaintained python2. If we want to save it, considerable
amount of work need to be done.

Details on the proposed removal of ibus-pinyin are as follows:

The default pinyin input method for ibus used to be ibus-pinyin[1] and its
upstream is https://github.com/ibus/ibus-pinyin/ . However, in recent years
the development shifts towards the libpinyin project, which provides ibus-
libpinyin[2] at https://github.com/libpinyin/ibus-libpinyin . The old ibus-
pinyin package was deprecated but not removed since someone may still prefer
to use it.

However, ibus-pinyin depends on libpyzy[3], a conversion library between
pinyin and bopomofo (a.k.a. zhuyin). This library has a dead upstream at
https://github.com/hsumita/libpyzy , which was exported from googlecode and
has no activity in the last 10 years. This library heavily depends on some
python2 scripts.

Now that the ongoing Python2 removal transition[4] is likely to complete
within the Bullseye cycle (you may find a progress chart here[5]), it's likely
that the libpyzy library will need to be ported to Python3 or dropped together
with ibus-pinyin, perhaps later this year.

I'm wondering if anyone is interested in porting python2 scripts in libpyzy
project. I had a really quick glance and have an estimation that it will
probably requires 5 hours of porting and another 15 hours of testing to ensure
a proper python3 migration if the developer has good experience in Python2->3
porting. Of course, another option is that we completely throw this library as
well as ibus-pinyin away.

P.S. I discussed it with the Arch Linux maintainer of ibus-pinyin/pyzy back in
2016 and 2018; the conclusion was to keep it at that time. Things have changed
since then.

Let me know if you have any thoughts. Volunteering would be highly welcome.

--
Regards,
Boyuan Yang

[1] https://tracker.debian.org/pkg/ibus-pinyin
[2] https://tracker.debian.org/pkg/ibus-libpinyin
[3] https://tracker.debian.org/pkg/libpyzy
[4] https://release.debian.org/transitions/html/python2-rm.html
[5] http://sandrotosi.me/debian/py2removal/py2removal_progress.png

[Gunnar Hjalmarsson]

On 2020-04-29 05:51, Boyuan Yang wrote:
> TL;DR: Let's drop package ibus-pinyin from Debian since it implicitly
> (build- )depends on the unmaintained python2. If we want to save it,
> considerable amount of work need to be done.

FWIW I want to mention that porting ibus-pinyin itself to py3 isn't very
hard. I did that last year as an experiment:

https://launchpad.net/~gunnarhj/+archive/ubuntu/ibus-pinyin/+packages

But the main obstacle seems to be libpyzy, which I have never looked at.

--
Gunnar Hjalmarsson
https://launchpad.net/~gunnarhj

[Kentaro Hayashi]

Hi,

FYI: I may be wrong because I didn't know well, but
It seems that someone tried to port libpyzy scripts to python3.

https://github.com/pyzy/pyzy/pull/4

Regards,

2020年4月29日(水) 15:06 Gunnar Hjalmarsson <gunn...@ubuntu.com>:

--
Kentaro Hayashi <ken...@gmail.com>

[Gunnar Hjalmarsson]

On 2020-04-29 08:37, Kentaro Hayashi wrote:
> FYI: I may be wrong because I didn't know well, but It seems that
> someone tried to port libpyzy scripts to python3.
>
> https://github.com/pyzy/pyzy/pull/4

You are not wrong at all; the work has already been done, apparently.

I tested it successfully and committed it as a patch to our repo.

https://salsa.debian.org/input-method-team/libpyzy/-/commit/40dc6196

So it looks like ibus-pinyin can stay in the archive for now. :)

[Boyuan Yang]

Hi,

在 2020-05-03星期日的 00:13 +0200，Gunnar Hjalmarsson写道：

> On 2020-04-29 08:37, Kentaro Hayashi wrote:
> > FYI: I may be wrong because I didn't know well, but It seems that
> > someone tried to port libpyzy scripts to python3.
> >
> > https://github.com/pyzy/pyzy/pull/4
>
> You are not wrong at all; the work has already been done, apparently.
>
> I tested it successfully and committed it as a patch to our repo.
>
> https://salsa.debian.org/input-method-team/libpyzy/-/commit/40dc6196
>
> So it looks like ibus-pinyin can stay in the archive for now. :)

The package may not FTBFS, but the python script did not run successfully.
There are trackbacks like this:

make[5]: Entering directory '/<<PKGBUILDDIR>>/data/db/android'
\
rm -f android.db; \
./create_db.py ./rawdict_utf16_65105_freq.txt | /usr/bin/sqlite3 android.db ||
\
( rm -f android.db ; exit 1 )
Traceback (most recent call last):
File "./create_db.py", line 83, in <module>
main()
File "./create_db.py", line 80, in main
create_db(sys.argv[1])
File "./create_db.py", line 51, in create_db
records = list(read_phrases(filename))
File "./create_db.py", line 19, in read_phrases
buf = file(filename).read()
NameError: name 'file' is not defined

I continued with more patching work to fix this certain issue
https://salsa.debian.org/input-method-team/libpyzy/-/commit/736051f06ecb41aada4448f6b89413d5b18bfb46
but that is not quite complete; it's still raising exceptions elsewhere. I'd
expect more help from others on making this script runnable.

--
Thanks,
Boyuan Yang

[Gunnar Hjalmarsson]

On 2020-05-03 03:46, Boyuan Yang wrote:
> 在 2020-05-03星期日的 00:13 +0200，Gunnar Hjalmarsson写道：

>> https://salsa.debian.org/input-method-team/libpyzy/-/commit/40dc6196
>>
>> So it looks like ibus-pinyin can stay in the archive for now. :)
>
> The package may not FTBFS, but the python script did not run successfully.
> There are trackbacks like this:
>
> make[5]: Entering directory '/<<PKGBUILDDIR>>/data/db/android'
> \
> rm -f android.db; \
> ./create_db.py ./rawdict_utf16_65105_freq.txt | /usr/bin/sqlite3 android.db ||
> \
> ( rm -f android.db ; exit 1 )
> Traceback (most recent call last):
> File "./create_db.py", line 83, in <module>
> main()
> File "./create_db.py", line 80, in main
> create_db(sys.argv[1])
> File "./create_db.py", line 51, in create_db
> records = list(read_phrases(filename))
> File "./create_db.py", line 19, in read_phrases
> buf = file(filename).read()
> NameError: name 'file' is not defined
>
> I continued with more patching work to fix this certain issue
> https://salsa.debian.org/input-method-team/libpyzy/-/commit/736051f06ecb41aada4448f6b89413d5b18bfb46
> but that is not quite complete; it's still raising exceptions elsewhere. I'd
> expect more help from others on making this script runnable.

Ouch, apparently it was too good to be true. :(

As regards that sort() function, I made an attempt in this MR:

https://salsa.debian.org/input-method-team/libpyzy/-/merge_requests/1

But then it really FTBFS:

Error: near line 21: unrecognized token: "\"
Error: near line 22: unrecognized token: "\"

(and tens of thousands of similar messages...)

It should be noted, though, that I get those error messages also by
simply commenting the sort() function, so they seem to be caused by code
running after that function.

As a side note I can mention that it also FTBFS for me due to this commit:

https://salsa.debian.org/input-method-team/libpyzy/-/commit/b1d8ffe7

[Gunnar Hjalmarsson]

I asked the author of the first patch, Martin Jansa, to take a look.

https://github.com/shr-project/pyzy/pull/1

Let's see.

(Anybody with the right skill is of course encouraged to step in and
help out.)