Bug#1006917: kpcli: "not well-formed (invalid token)" when opening a file

[Rhonda D'Vine]

Package: kpcli
Version: 3.1-3.1
Severity: grave
Tags: upstream
Justification: causes serious data loss

Dear Maintainer,

I store my passwords in a keepass file that I exclusively use through kpcli.
After the last kernel upgrade reboot I was unable to open the file anymore, and
thus can't access my passwords. I have an aged backup, and most sites offer
password resets, but this is actually a serious data loss.

When I try to open the database now I get the following error message:

➤ kpcli --kdb rhonda.kdbx
Please provide the master password: *************************
Couldn't load the file rhonda.kdbx:
not well-formed (invalid token) at line 3103, column 15, byte 100409 at /usr/lib/x86_64-linux-gnu/perl5/5.34/XML/Parser.pm line 187.

So I have somehow the hope that the data isn't lost completely, only that the
XML parser is stumbling upon something. I haven't had the nerve yet to dig
further into it and try to unpack the whole situation, make kpcli dump what it
gives to XML::Parser, that part gives me a bit of a hope because it clearly can
decrypt the file in the first place, but it makes it unusable to the
"innocent".

If you are able to give me any helping hand on those grounds, they would be
very much appreciated! Because as it stands I assume this might happen to
others, and I'm uncertain if it would have anything to do with specific data
stored in some comment or password field or whatever.

Thanks in advance,
Rhonda


-- System Information:
Debian Release: bookworm/sid
APT prefers testing
APT policy: (500, 'testing')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.16.0-3-amd64 (SMP w/8 CPU threads; PREEMPT)
Locale: LANG=de_AT.UTF-8, LC_CTYPE=de_AT.UTF-8 (charmap=UTF-8), LANGUAGE=de_AT:de
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages kpcli depends on:
ii libclone-perl 0.45-1+b2
ii libcrypt-rijndael-perl 1.16-1+b1
ii libfile-keepass-perl 2.03-1.1
ii libsort-naturally-perl 1.03-2
ii libterm-readkey-perl 2.38-1+b3
ii libterm-readline-gnu-perl 1.42-2+b1
ii libterm-shellui-perl 0.92-4
ii perl 5.34.0-3

Versions of packages kpcli recommends:
ii libcapture-tiny-perl 0.48-1
ii libclipboard-perl 0.27-1
pn libdata-password-perl <none>
pn libmath-random-isaac-perl <none>

kpcli suggests no packages.

-- no debconf information

[Rhonda D'Vine]

Yes indeed, i had to fix it through the module. Sorry that I wasn't clear on that part. Likely this should be changed to be a bug in the module interface since the frontend shouldn't have to know too much about what's allowed or not in the fields, the module should give the frontend error messages accordingly, but I hadn't had the time to look up if that's possible to differentiate.

Thanks for asking for clarification,
Rhonda

Am 8. März 2022 16:47:41 MEZ schrieb Lester Hightower <lester.h...@gmail.com>:

Hi Rhonda,

I am happy that you found and fixed your problem. I suspect, however, that the code that you changed was not actually kpcli code but, instead, File::KeePass code -- the module that kpcli uses to read and write keepass files. https://metacpan.org/pod/File::KeePass

Can you confirm that I am correct about that?

Thanks,

--
Lester

On Tue, Mar 8, 2022 at 10:33 AM Rhonda D'Vine <rho...@deb.at> wrote:

  Hi,

$buffer =~ s/\e//g;

 .. this was all that was needed to fix my mess.  Though, kpcli for
obvious reasons shouldn't be able to write broken data it can't read
again, so I keep seeing this as a severe bug in the code which can lead
to data loss for people who aren't familiar enough with perl or who
don't have friends who support them to dig down the issue.

 The above line was a quick fix for my case, I'm uncertain if it might
appear to others in other ways, but this clearly goes against the
principle of robustness.

 Upstream is at 3.6 in the meantime, I'm willing to update it now that I
digged a bit further into it.  If I don't hear back in the next few days
I propose an NMU for it, as thanks for having it around in the first
place. :)

 Enjoy,
Rhonda [happy again]


* Rhonda D'Vine <rho...@deb.at> [2022-03-08 16:19:46 CET]:
>    Hi,
>
>  I managed to find the culprit With A Little Help From My Friends[tm]. I
> used Data::Dumper before the content got passed to XML::Parser, and it
> turned out that there is an Escape character (0x1b, ^[) in a comment
> field.
>
>  kpcli seems to have accepted this when the comment was pasted and
> stored it happily, but was unable to re-read the file written with that
> in it.
>
>  I'm currently fiddling around to delete that escape character on load
> time and have kpcli start, allowing me to save it without the escape
> character, hopefully allowing to re-read it afterwards.
>
>  I'll keep you posted,
> Rhonda

--
Fühlst du dich mutlos, fass endlich Mut, los      |
Fühlst du dich hilflos, geh raus und hilf, los    | Wir sind Helden
Fühlst du dich machtlos, geh raus und mach, los   | 23.55: Alles auf Anfang
Fühlst du dich haltlos, such Halt und lass los    |

--
Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[Arno Töll]

Hi Rhonda,

Am 08.03.22 um 16:31 schrieb Rhonda D'Vine:

> Upstream is at 3.6 in the meantime, I'm willing to update it now that I
> digged a bit further into it. If I don't hear back in the next few days
> I propose an NMU for it, as thanks for having it around in the first
> place. :)

please feel free to do, and go ahead. Feel free to add yourself as a
maintainer/uploader if you wish. ;-)

The issue has been properly reassigned in the meantime. Thanks for that
Lester.


--

Arno Töll

[Rhonda D'Vine]

* Arno Töll <ar...@debian.org> [2022-03-17 14:07:02 CET]:

> Hi Rhonda,
>
> Am 08.03.22 um 16:31 schrieb Rhonda D'Vine:
> > Upstream is at 3.6 in the meantime, I'm willing to update it now that I
> > digged a bit further into it. If I don't hear back in the next few days
> > I propose an NMU for it, as thanks for having it around in the first
> > place. :)
>
> please feel free to do, and go ahead. Feel free to add yourself as a
> maintainer/uploader if you wish. ;-)

Do you have a copy of the git repository you used still around? It
never seems to have been moved to salsa, and I for obvious reasons would
work based on what's there already. :)

> The issue has been properly reassigned in the meantime. Thanks for that
> Lester.

It actually hasn't been reassigned but closed I noticed, and I'm also
not so convinced to call it only a minor issue, because as I explained,
I managed to fix it because I know my way around the code, but that's
not something to expect from regular users. I will be looking into
filing this with the upstream tracker though.

So long,

[Lester Hightower]

Rhonda,

Please note that marking this bug as "grave" queued kpcli for autoremoval from Debian testing:

kpcli 3.1-3.1 is marked for autoremoval from testing on 2022-04-06

It is affected by these RC bugs:
1006917: kpcli: "not well-formed (invalid token)" when opening a file
 https://bugs.debian.org/1006917

This mail is generated by:
https://salsa.debian.org/release-team/release-tools/-/blob/master/mailer/mail_autoremovals.pl

Autoremoval data is generated by:

https://salsa.debian.org/qa/udd/-/blob/master/udd/testing_autoremovals_gatherer.pl 

Receiving that notice is what made me act yesterday.

--
Lester

[Arno Töll]

clone 1006917 -1
reassign -1 libfile-keepass-perl
retitle -1 libfile-keepass-perl: crashes "not well-formed (invalid token)" when finding escape characters
severity -1 important
thanks

Hey,

Am 18.03.22 um 12:02 schrieb Rhonda D'Vine:

* Arno Töll <ar...@debian.org> [2022-03-17 14:07:02 CET]:

Hi Rhonda,

Am 08.03.22 um 16:31 schrieb Rhonda D'Vine:

  Upstream is at 3.6 in the meantime, I'm willing to update it now that I
digged a bit further into it.  If I don't hear back in the next few days
I propose an NMU for it, as thanks for having it around in the first
place. :)

please feel free to do, and go ahead. Feel free to add yourself as a
maintainer/uploader if you wish. ;-)

 Do you have a copy of the git repository you used still around?  It
never seems to have been moved to salsa, and I for obvious reasons would
work based on what's there already. :)

Alioth's archive of the repository is at https://alioth-archive.debian.org/git/collab-maint/kpcli.git.tar.xz. That allows for bare import, including git history into salsa.

Unfortunately I don't have a lot of time for Debian these days, sorry about that.

The issue has been properly reassigned in the meantime. Thanks for that
Lester.

 It actually hasn't been reassigned but closed I noticed, and I'm also
not so convinced to call it only a minor issue, because as I explained,
I managed to fix it because I know my way around the code, but that's
not something to expect from regular users.  I will be looking into
filing this with the upstream tracker though.

How about duplicating the issue and reassigning one to libfile-keepass-perl? I'm not sure about the priority, but something below RC might do for that. I did so as per this mail.

-- 
Arno Töll

[Rhonda D'Vine]

reopen 1007914
thanks

* Lester Hightower <lester.h...@gmail.com> [2022-03-18 12:53:30 CET]:

> Please note that marking this bug as "grave" queued kpcli for autoremoval
> from Debian testing:

I am very well aware how the bug states work. Thing is, why do you
think the data loss isn't severe enough to warrant a release critical
status? It's definitely not a minor issue.

> Receiving that notice is what made me act yesterday.

Thing is, demoting release critical bugs without fixing them isn't the
most helpful thing. I know that it might be a pain at times, but not
being able to get to your passwords is a very critical issue for kpcli.

That said, even having it in a release-critical state against the
library package would remove kpcli because it would get removed together
with it, and given that kpcli is the only package depending on
libfile-keepass-perl the difference is only minor in the end.

Cheers,

[Lester Hightower]

I do not equate your one, esoteric data access problem with justification for removing the package from Debian. There is no security problem and no data was lost. Even if you had not fixed the problem in File::KeePass yourself, there are many other programs that operate on KeePass files that could have been used to access your data.

--
Lester

[Rhonda D'Vine]

Hi,

* Lester Hightower <lester.h...@gmail.com> [2022-03-18 16:12:45 CET]:

> I do not equate your one, esoteric data access problem with justification
> for removing the package from Debian.

Pasting data into the comment field of an entry is nothing I would
anywhere closely consider esoteric, rather the opposite. And that a
tool would write data out that it couldn't read back in is something
that is utterly confusing, to say the least, and a clear bug that is not
just annoying but can impact people's access. That it was easy to fix
doesn't reduce the impact of the issue.

> There is no security problem and no data was lost. Even if you had not
> fixed the problem in File::KeePass yourself, there are many other
> programs that operate on KeePass files that could have been used to
> access your data.

This is where you are clearly wrong. I tried opening the file with
other keepass tools, and it boiled down to the same issue: There was
data in the XML that weren't valid, and thus couldn't get parsed by any
keepass tool.

Please don't try to reason with things that aren't the case.