Bug#998105: backintime-qt: Fails to access the keyring with python3-keyring 23.2.0-1

[Sven Geuer]

Package: backintime-qt
Version: 1.2.1-3
Severity: normal

Dear Maintainer,

In ssh mode backintime fails to read previously stored credentials since
python3-keyring 23.2.0-1 has been rolled out. Instead it always calls
backintime-askpass to query the password from the GUI.

Logs say it cannot work with the back end python3-keyring uses:
keyring.backends.chainer

$ backintime-qt --debug
DEBUG: [common/backintime.py:583 argParse] Arguments: {'debug': True} |
unknownArgs: []

Back In Time
Version: 1.2.1

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

DEBUG: [common/backintime.py:670 getConfig] config file:
/home/sg/.config/backintime/config
DEBUG: [common/backintime.py:671 getConfig] share path:
/home/sg/.local/share/backintime
DEBUG: [common/backintime.py:672 getConfig] profiles: 1=Hauptprofil
DEBUG: [common/pluginmanager.py:90 PluginManager.load] Register plugin path
/usr/share/backintime/plugins
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin
qt4plugin.py
DEBUG: [common/pluginmanager.py:106 PluginManager.load] Add plugin
notifyplugin.py
Warning: Ignoring XDG_SESSION_TYPE=wayland on Gnome. Use
QT_QPA_PLATFORM=wayland to run on Wayland anyway.
DEBUG: [common/tools.py:823 keyringSupported] No appropriate keyring found.
'keyring.backends.chainer' can't be used with BackInTime
[...]

With python3-keyring 22.0.1-1 backintime succeeds to read the ssh password from
the keyring (no password dialog pops up).

However logs now say

[...]
DEBUG: [common/tools.py:823 keyringSupported] No appropriate keyring found.
'keyring.backends.SecretService' can't be used with BackInTime
[...]

Further investigations showed that one can force python3-keyring 23.2.0-1 to
use keyring.backends.SecretService instead of keyring.backends.chainer.

~/.config/python_keyring/keyringrc.cfg:

[backend]
default-keyring=keyring.backends.SecretService.Keyring


Doing so apparently allows backintime to read from the keyring (no password
dialog pops up), but still the logs show

[...]
DEBUG: [common/tools.py:823 keyringSupported] No appropriate keyring found.
'keyring.backends.SecretService' can't be used with BackInTime
[...]

Two Conclusion:

1) Backintime fails to access the keyring when keyring.backends.chainer is in
use.
2) Backintime's test for keyring back ends does not work properly. It does not
recognize a back end as functional it does work with later on.


-- System Information:
Debian Release: bookworm/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 5.14.0-2-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=de_DE.UTF-8, LC_CTYPE=de_DE.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages backintime-qt depends on:
ii backintime-common 1.2.1-3
ii libnotify-bin 0.7.9-3
ii policykit-1 0.105-31
ii python3 3.9.2-3
ii python3-dbus.mainloop.pyqt5 5.15.4+dfsg-4
ii python3-pyqt5 5.15.4+dfsg-4
ii x11-utils 7.7+5

Versions of packages backintime-qt recommends:
ii python3-secretstorage 3.3.1-1

Versions of packages backintime-qt suggests:
pn meld | kompare <none>

-- no debconf information

[BiT]

> DEBUG: [common/tools.py:823 keyringSupported] No appropriate keyring found.
> 'keyring.backends.chainer' can't be used with BackInTime
> ...

> Two Conclusion:
> 1) Backintime fails to access the keyring when keyring.backends.chainer is in use.

Correct.

The keyring.backends.chainer.chainerBackend is not yet supported in BiT
and perhaps never will be because it iterates over all installed backends
and picks the "best one" by some internal logic.

The chosen backend my not be the one where the users saves the credentials anyhow.

So a decent fix for this would be to add a settings GUI to BiT so that
the user can choose which backend is the right one (containing the credentials).

I have documented future implementation options in this issue comment:

https://github.com/bit-team/backintime/issues/1321#issuecomment-1271689827

> With python3-keyring 22.0.1-1 backintime succeeds to read the ssh password

> However logs now say:

> DEBUG: [common/tools.py:823 keyringSupported] No appropriate keyring found.
> 'keyring.backends.SecretService' can't be used with BackInTime
> ...
> Two Conclusion:

> ...

> 2) Backintime's test for keyring back ends does not work properly.
> It does not recognize a back end as functional it does work with later on.

Correct. I think this has the same reason as the bug #1321 linked in above issue.

I am preparing a fix for this bug (but NOT adding support for the ChainerBackend!)
and hopefully attach a patch for this if the package maintainer really wants
to backport this to BiT v1.2.1 (with no support from the BiT team I guess
since we are preparing a new stabilized release 1.3.x where many bugs are fixed
but no new features are added (therefore "stabilize" ;-)

So I'd prefer if the package maintainer waited for stabilized BiT version
and packages the new BiT version we are working on.

[c.b...@posteo.jp]

Dear Sven,

there is a new release 1.3.3 in "unstable" branch of Debian.

Can you please try to reproduce the problem with that version and then
report back.

Thanks
Christian

[Sven Geuer]

Control: reopen -1 =

Hello Christian,

Unfortunately BiT version 1.3.3 still fails with a version of python3-
keyring offering keyring.backends.chainer.

Debug output:

$ backintime-qt --debug
DEBUG: [common/backintime.py:589 argParse] Arguments: {'debug': True} | unknownArgs: []

Back In Time
Version: 1.3.3

Back In Time comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; type `backintime --license' for details.

DEBUG: [common/backintime.py:677 getConfig] config file: /home/sg/.config/backintime/config
DEBUG: [common/backintime.py:678 getConfig] share path: /home/sg/.local/share/backintime
DEBUG: [common/backintime.py:679 getConfig] profiles: 1=Hauptprofil
DEBUG: [common/pluginmanager.py:240 PluginManager.load] Register plugin path /usr/share/backintime/plugins
DEBUG: [common/pluginmanager.py:257 PluginManager.load] Add plugin qt4plugin.py
DEBUG: [common/pluginmanager.py:257 PluginManager.load] Add plugin notifyplugin.py
QSocketNotifier: Can only be used with threads started with QThread
DEBUG: [qt/qttools.py:175 createQApplication] QT QPA platform plugin: wayland
DEBUG: [qt/qttools.py:176 createQApplication] QT_QPA_PLATFORMTHEME=gnome
DEBUG: [qt/qttools.py:178 createQApplication] QT_STYLE_OVERRIDE=<not set>
DEBUG: [qt/qttools.py:179 createQApplication] QT active style: fusion
DEBUG: [qt/qttools.py:180 createQApplication] QT fallback style: Adwaita
DEBUG: [qt/qttools.py:181 createQApplication] QT supported styles: ['Windows', 'Fusion']
DEBUG: [qt/qttools.py:182 createQApplication] themeSearchPaths: ['/usr/share/icons', ':/icons']
DEBUG: [qt/qttools.py:183 createQApplication] fallbackSearchPaths: []
DEBUG: [qt/qttools.py:185 createQApplication] Is SystemTray available: False
DEBUG: [qt/qttools.py:197 createQApplication] Trying to set App ID for non-privileged user
DEBUG: [common/tools.py:862 keyringSupported] Available keyring backends:
DEBUG: [common/tools.py:865 keyringSupported] keyring.backends.fail.Keyring (priority: 0)
DEBUG: [common/tools.py:865 keyringSupported] keyring.backends.chainer.ChainerBackend (priority: 10)
DEBUG: [common/tools.py:865 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)
DEBUG: [common/tools.py:865 keyringSupported] keyring.backends.libsecret.Keyring (priority: 4.8)
DEBUG: [common/tools.py:878 keyringSupported] Metaclass keyring.backends.Gnome.Keyring not found: AttributeError("module 'keyring.backends' has no attribute 'Gnome'")
DEBUG: [common/tools.py:880 keyringSupported] Metaclass keyring.backends.kwallet.Keyring not found: AttributeError("module 'keyring.backends.kwallet' has no attribute 'Keyring'")
DEBUG: [common/tools.py:884 keyringSupported] Metaclass keyring.backend.SecretServiceKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'SecretServiceKeyring'")
DEBUG: [common/tools.py:886 keyringSupported] Metaclass keyring.backend.GnomeKeyring not found: AttributeError("module 'keyring.backend' has no attribute 'GnomeKeyring'")
DEBUG: [common/tools.py:888 keyringSupported] Metaclass keyring.backend.KDEKWallet not found: AttributeError("module 'keyring.backend' has no attribute 'KDEKWallet'")
DEBUG: [common/tools.py:899 keyringSupported] Available supported backends: [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>]
DEBUG: [common/tools.py:905 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTime
[...]

I still need to pin python3-keyring to use
keyring.backends.SecretService.

Sven

--
GPG Fingerprint
3DF5 E8AA 43FC 9FDF D086 F195 ADF5 0EDA F8AD D585

[BiT dev]

On Tue, 21 Feb 2023 20:40:21 +0100 Sven Geuer <debm...@g-e-u-e-r.de> wrote:

> I still need to pin python3-keyring to use
> keyring.backends.SecretService.

What do you mean with "pin"? Installing an old version of python3-keyring (without the chainer) by pinning the ("old") DEB package (version)?

> Unfortunately BiT version 1.3.3 still fails with a version of python3-
> keyring offering keyring.backends.chainer.

> [...]

TLDR:
-----

"chainer" is new and a "meta keyring" that decides internally which actually installed
keyring will be used using some heuristics.

"chainer" is not supported by BiT since the heuristics may choose another keyring in the future for any reason
and the wrong keyring is used then (which does not contain the stored credentials for BiT).

To solve (work around) this problem add a keyring config file to specify ("pin") which keyring you want
to use instead of chainer, see these instructions here:

https://github.com/bit-team/backintime#non-working-password-safe-and-bit-forgets-passwords-keyring-backend-issues

-> Just use your prefered keyring in the config file:

default-keyring=keyring.backends.SecretService.Keyring



In full length:
---------------

To really use a a keyring two things must be installed:

1. A supported keyring software (debian package)
2. A python library that has a "driver" to enable the keyring of 1. for python

The BiT logs show that "secret service" is installed (satisfies above number 1.):

> DEBUG: [common/tools.py:862 keyringSupported] Available keyring backends:

...

> DEBUG: [common/tools.py:865 keyringSupported] keyring.backends.SecretService.Keyring (priority: 5)

The BiT logs also show that above number 2. is satisfied
("python3-secretstorage" is installed together with "python3-keyring" AFAIR since it has a "depends" relationship):

> DEBUG: [common/tools.py:899 keyringSupported] Available supported backends:
> [<class 'keyring.backends.SecretService.Keyring'>, <class 'keyring.backends.kwallet.DBusKeyring'>]

Since newer "python3-keyring" versions use "chainer" by default now (which is not supported by BiT) no keyring is available in BiT:

> DEBUG: [common/tools.py:905 keyringSupported] No appropriate keyring found. 'keyring.backends.chainer' can't be used with BackInTime

Outlook:
--------------

I think I should add a link to above "known issues" section to the debug output.

How the chainer issue could be fixed is described quite well in this feature request (add a GUI to select the keyring):

https://github.com/bit-team/backintime/issues/1330

[Sven Geuer]

Hi BiT Dev,

thank you for the detailled reply.

More inline below ...

On Thu, 2023-02-23 at 01:53 +0100, BiT dev wrote:
> On Tue, 21 Feb 2023 20:40:21 +0100 Sven Geuer <debm...@g-e-u-e-r.de>
> wrote:
>
> > I still need to pin python3-keyring to use
> > keyring.backends.SecretService.
>
> What do you mean with "pin"? Installing an old version of python3-
> keyring (without the chainer) by pinning the ("old") DEB package
> (version)?

With "pin" I meant configuring python3-keyring per

~/.config/python_keyring/keyringrc.cfg:

[backend]
default-keyring=keyring.backends.SecretService.Keyring

More below ...

I understand this all very well. The current situation just does not
deem user friendly to me. I am afraid the average user is unable to
detect what the problem is and how to fix it, especially as things
worked out of the box before python3-keyring 23.2.0-1 was introduced.

>
> Outlook:
> --------------
>
> I think I should add a link to above "known issues" section to the
> debug output.
>
> How the chainer issue could be fixed is described quite well in this
> feature request (add a GUI to select the keyring):
>
> https://github.com/bit-team/backintime/issues/1330
>

This may help, although it still requires sufficient insight on the
part of the user.

Would it be feasible to probe the supported backends for already stored
BiT credentials and choose the matching backend automatically?

Love to hear what you think.

[Jürgen Altfeld]

On Thu, 23 Feb 2023 21:38:55 +0100 Sven Geuer <debm...@g-e-u-e-r.de> wrote:

> The current situation just does not deem user friendly to me.
> I am afraid the average user is unable to detect what the problem is and how to fix it,
> especially as things worked out of the box before python3-keyring 23.2.0-1 was introduced.

I fully agree with that!

> Would it be feasible to probe the supported backends for already stored
> BiT credentials and choose the matching backend automatically?

TLDR;
----

I propose that I open a new issue at BiT and fix this bug ASAP by enabling ChainerBackend support in BiT.
I had already documented and prepared this
fix three months ago (but the code is still commented out):

https://github.com/bit-team/backintime/blob/e22c7f253fa048fab9119f4cbea34c0fa8d1aba6/common/tools.py#L889-L897


Details:
-------

To give a little bit context to this issue with the "chainer" backend:

I (nickname "aryoda" at GH) have fixed some non-chainer-related keyring issues in BiT and discovered
during my debug sessions the existence of the new "ChainerBackend" and its impact.

Since we (the BiT developers) are currently only fixing bugs to stabilize
the new BiT release I did not want add support for another new and untested password safe backend
so I have just documented a work-around and requested a new feature at the keyring
homepage (https://github.com/jaraco/keyring/issues/602) together with a feature request
for BiT ("add a GUI to choose a backend", see https://github.com/bit-team/backintime/issues/1330).

I now realize (thanks to your persistence & asking the right questions - excellent soft skills BTW :-)
I have totally under-estimated the impact of this keyring change:

The current and all old BiT versions do **NOT** support the ChainerBackend
(the supported backends are white-listed in BiT) which is now the default
in the new keyring package and I see a storm of new issues in front of us ;-)

So to come back to your question ("choose the matching backend automatically"):

This could possibly be implemented in BiT but

- it would be invisible to users again until we take this as default and
add the GUI to allow the user to change this

- an "old" password in an previously used password safe could "win the election"

- quite a similar logic is already implemented in the ChainerBackend in `keyring` to get a password:
https://github.com/jaraco/keyring/blob/7ffff5b285d6f9d9b8cba33a9d5e9071cdd507b9/keyring/backends/chainer.py#L47-L51

So I propose that I open a new issue at BiT and fix this bug ASAP by enabling ChainerBackend support in BiT.
Interestingly I had already documented and prepared this fix three months ago:

https://github.com/bit-team/backintime/blob/e22c7f253fa048fab9119f4cbea34c0fa8d1aba6/common/tools.py#L889-L897

BTW: I see one key difference between your proposal (to recognize the best keyring backend in BiT)
and the ChainerBackend:

Password updates ("set_password"): ChainerBackend uses the first (highest-rated) backend in the chain
while in BiT we could update an existing password in the same backend where the old password exists
(may not be the same backend chosen by ChainerBackend!).

https://github.com/jaraco/keyring/blob/7ffff5b285d6f9d9b8cba33a9d5e9071cdd507b9/keyring/backends/chainer.py#L53-L58

Still implementing this advanced password update logic requires too much dev & testing time now so I would rely
on the ChainerBackend for now and observe how users come back with issues (if any) in the future...

So:

1. Would you agree with my proposal to fix it (just enabling ChainerBackend in BiT)?
Do you have other ideas/suggestions?
2. Are you able (= still allowed) to apply a patch to the upcoming BiT package ("unstable" AFAIR) using my fix?

THX a lot for your support!

[Jürgen Altfeld]

I have opened a new issue at BiT:

https://github.com/bit-team/backintime/issues/1410

Would it be possible to continue or conversation about this bug there
to make it visible for others (= non-Debian users of BiT)?

THX :-)

[Sven Geuer]

See inline below ...

On Fri, 2023-02-24 at 16:26 +0100, Jürgen Altfeld wrote:
[...]

>
> I now realize (thanks to your persistence & asking the right
> questions - excellent soft skills BTW :-)

Thanks for the kudos :-)

[...]

>
> 1. Would you agree with my proposal to fix it (just enabling
> ChainerBackend in BiT)?
>    Do you have other ideas/suggestions?

That's an excellent proposal. To me it seems the best solution.

> 2. Are you able (= still allowed) to apply a patch to the upcoming
> BiT package ("unstable" AFAIR) using my fix?

Bookworm is currently in Soft Freeze, so targeted fixes are still
possible. As I am only the reporter of the bug the maintainer, Jonathan
Wiltshire, needs to make sure the patch goes into BiT in time.
According to https://release.debian.org/testing/freeze_policy.html#hard
2023-03-12 is the deadline.

>
> THX a lot for your support!
>

My pleasure!

[BiT dev]

I have implemented the support for the ChainerBackend class of the python keyring package
and it is currently on its way into the dev branch of BiT via this pull request:

https://github.com/bit-team/backintime/pull/1413


Any feed-back (esp. testing it on your computer) is welcome. THX a lot!

[Sven Geuer]

On Fri, 2023-03-03 at 01:46 +0100, BiT dev wrote:
> [...]
> https://github.com/bit-team/backintime/pull/1413
> [...]

I created a patch to backintime 1.3.3-4 and tested it successfully.

Here is the merge request for a backintime 1.3.3-5:
https://salsa.debian.org/jmw/pkg-backintime/-/merge_requests/9

Cheers,
Sven

[c.b...@posteo.jp]

Dear Sven,

this is one of the upstream maintainers (@buhtz).

Thanks a lot for your contribution. Keep in mind the the upstream PR
isn't merged yet.

And wouldn't it be better to use the next upstream release instead of a
Debian specific patch?

We have no problem to prepare a upstream 1.3.4 if Jonathon (or Fabio)
tell us that this is used for Debian 12 and not blocked because of the
current freeze.

We'll wait for advice.

Kind
Christian Buhtz

[Sven Geuer]

Hi Christian,

On Sun, 2023-03-05 at 19:18 +0000, c.b...@posteo.jp wrote:
> Dear Sven,
>
> this is one of the upstream maintainers (@buhtz).
>
> Thanks a lot for your contribution. Keep in mind the the upstream PR
> isn't merged yet.
>
> And wouldn't it be better to use the next upstream release instead of
> a Debian specific patch?

A new upstream release would definitely be the better approach.

>
> We have no problem to prepare a upstream 1.3.4 if Jonathon (or Fabio)
> tell us that this is used for Debian 12 and not blocked because of
> the
> current freeze.
>
> We'll wait for advice.

I just wanted to offer what I did anyway to test in Debian the changes
the PR would bring. Let's see how Jonathan (or Fabio) wants to proceed
with this bug.

Cheers,
Sven

[c.b...@posteo.jp]

Do I see it correct that the commit was done but not yet uploaded?

In result there should be a 1.3.3-5, right?

[Fabio Fantoni]

Il 29/03/2023 10:11, c.b...@posteo.jp ha scritto:
> Do I see it correct that the commit was done but not yet uploaded?
>
> In result there should be a 1.3.3-5, right?
>

 There is still no reply  from Jonathan Wiltshire, I also sent him a
mail some days ago.

I'll probably try to do a NMU in the weekend