Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss

Re: Appstream data not in UTF-8?

2 views
Skip to first unread message

Raphael Hertzog

unread,
Dec 7, 2023, 5:10:05 AM12/7/23
to
On Thu, 07 Dec 2023, Raphael Hertzog wrote:
> tracker.debian.org has been failing to import the appstream metadata for a
> while (since November 26th) with this exception:

Quick correction. The first time it failed that way was on November 19th
at 08:13 UTC.

Cheers,
--
⢀⣴⠾⠻⢶⣦⠀ Raphaël Hertzog <her...@debian.org>
⣾⠁⢠⠒⠀⣿⡁
⢿⡄⠘⠷⠚⠋ The Debian Handbook: https://debian-handbook.info/get/
⠈⠳⣄⠀⠀⠀⠀ Debian Long Term Support: https://deb.li/LTS

Raphael Hertzog

unread,
Dec 7, 2023, 5:10:05 AM12/7/23
to
Hello Matthias,

tracker.debian.org has been failing to import the appstream metadata for a
while (since November 26th) with this exception:

Task UpdateAppStreamStatsTask failed with the following traceback.

Traceback (most recent call last):
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/tasks/base.py", line 378, in run_task
task.execute()
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/tasks/base.py", line 256, in execute
call_methods_with_prefix(self, 'execute_')
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/utils/misc.py", line 44, in
call_methods_with_prefix
method(*args, **kwargs)
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/vendor/debian/tracker_tasks.py", line 889,
in execute_main
self._load_appstream_hint_stats(component, arch, all_stats)
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/vendor/debian/tracker_tasks.py", line 728,
in _load_appstream_hint_stats
hints_json = get_resource_text(url, force_update=self.force_update)
File "/srv/tracker.debian.org/distro-tracker/distro_tracker/core/utils/http.py", line 356, in
get_resource_text
return content.decode(encoding)

Exception Type: UnicodeDecodeError
Exception Value: 'utf-8' codec can't decode byte 0xcc in position 990416: invalid continuation byte
Request data not supplied


Can you look into this and fix the data? (the precise position of the
error in the stream varied over the days, here you have the position as of
today, December 7th 09:36 UTC)

This refers to those two lines:
url = 'https://appstream.debian.org/hints/sid/{}/Hints-{}.json.gz' \
.format(section, arch)
hints_json = get_resource_text(url, force_update=self.force_update)

arch is "amd64", an section is likely "main" but it's not impossible that
it's in one of the other sections.

Thank you in advance for your help!

Andreas Tille

unread,
Dec 7, 2023, 5:30:03 AM12/7/23
to
Am Thu, Dec 07, 2023 at 11:03:29AM +0100 schrieb Raphael Hertzog:
> On Thu, 07 Dec 2023, Raphael Hertzog wrote:
> > tracker.debian.org has been failing to import the appstream metadata for a
> > while (since November 26th) with this exception:
>
> Quick correction. The first time it failed that way was on November 19th
> at 08:13 UTC.

I do not remember the time exactly but my gut feeling says this is pretty
close to the bookworm upgrade.

Kind regards
Andreas.

--
http://fam-tille.de

Matthias Klumpp

unread,
Dec 8, 2023, 8:50:03 PM12/8/23
to
Hi!

Thanks for letting me know about the issue! Apparently some database
entry got corrupted, and concerningly I do not know why... I updated
the software though and refreshed the entry, and also added some UTF-8
sanitization for new entries, so hopefully this will stay fixed.
The issue should be gone already :-)

Best,
Matthias
0 new messages