Bug#1016516: notion: Notion fails on startup (in sid)

[Jurij Smakov]

Package: notion
Version: 4.0.2+dfsg-6
Severity: important

Greetings,

I ran the apt-get update yesterday on my laptop (running sid), and
now notion fails to start. After I type my username and password at
the DM prompt, it attempts to run it, but then returns back to the
prompt.

.xsession-errors looks like this:

Xsession: X session started for jurij at Tue Aug 2 08:36:18 AM IST 2022
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting DISPLAY=:0
dbus-update-activation-environment: setting XAUTHORITY=/home/jurij/.Xauthority
localuser:jurij being added to access control list
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
2022-08-02 08:36:18 INFO /notion/../ioncore.c:596: ioncore_startup: Starting Notion
notion: ../nptl/pthread_mutex_lock.c:424: __pthread_mutex_lock_full: Assertion `e != ESRCH || !robust' failed.

-- System Information:
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (500, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 5.18.0-3-amd64 (SMP w/12 CPU threads; PREEMPT)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages notion depends on:
ii libc6 2.33-8
ii libice6 2:1.0.10-1
ii liblua5.3-0 5.3.6-1
ii libreadline8 8.1.2-1.2
ii libsm6 2:1.2.3-1
ii libx11-6 2:1.8.1-1
ii libxext6 2:1.3.4-1
ii libxft2 2.3.4-1
ii libxinerama1 2:1.1.4-3
ii libxrandr2 2:1.5.2-2+b1
ii lxterminal [x-terminal-emulator] 0.4.0-2
ii menu 2.1.49
ii terminator [x-terminal-emulator] 2.1.1-1
ii x11-utils 7.7+5
ii xterm [x-terminal-emulator] 372-1

Versions of packages notion recommends:
ii xfonts-100dpi 1:1.0.4+nmu1.1
ii xfonts-75dpi 1:1.0.4+nmu1.1

Versions of packages notion suggests:
ii wmdocker 1.5-2

-- no debconf information

[Dima Kogan]

Jurij: thanks for the report. This issue is already tracked here:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016363#15

and here:

https://github.com/raboof/notion/issues/345

Arnout (upstream) can't reproduce it currently.

Jurij and Göran: since you can realiably see this breakage, can you get
a log to Arnout so that he can debug? One useful tool here would be the
rr debugger:

http://rr-project.org

You can save a trace of the crashing process, which can then be replayed
forwards and backwards later. "rr pack" would make the traces portable
so that they can be replayed on another machine.

[Jurij Smakov]

On Tue, Aug 2, 2022 at 6:32 PM Dima Kogan <dko...@debian.org> wrote:

Jurij: thanks for the report. This issue is already tracked here:

  https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016363#15

and here:

  https://github.com/raboof/notion/issues/345

Arnout (upstream) can't reproduce it currently.

Jurij and Göran: since you can realiably see this breakage, can you get
a log to Arnout so that he can debug? One useful tool here would be the
rr debugger:

  http://rr-project.org

I tried launching notion under rr, and then this assertion does not trigger, unfortunately.

 

You can save a trace of the crashing process, which can then be replayed
forwards and backwards later. "rr pack" would make the traces portable
so that they can be replayed on another machine.

--

Jurij Smakov | ju...@wooyd.org | Key ID: 43C30A7D

[Dima Kogan]

Jurij Smakov <ju...@wooyd.org> writes:

> I tried launching notion under rr, and then this assertion does not
> trigger, unfortunately.

That's too bad. Thanks for trying it out.

This issue is a race condition, and rr inherently limits the process to
one core. Can you try "rr record --chaos"? This randomizes the thread
switching, and is often effective and triggering these kinds of failures
under rr.

[Jurij Smakov]

Did not have much luck with --chaos either, the assertion does not trigger.

I was able to get a fully symbolized trace with gdb though, it can be found here: http://wooyd.org/dbg/notion.backtrace.txt

Does this help? It would probably be useful to get backtraces for other threads as well, but I was not able to quickly figure out how to get rid of this warning:

"warning: Unable to find libthread_db matching inferior's thread library, thread debugging will not be available."

which might be preventing me from accessing other threads' information.

[Göran Weinholt]

Jurij Smakov <ju...@wooyd.org> writes:

> On Wed, Aug 3, 2022 at 5:37 PM Dima Kogan <dko...@debian.org> wrote:
>
> Jurij Smakov <ju...@wooyd.org> writes:
>
> > I tried launching notion under rr, and then this assertion does not
> > trigger, unfortunately.
>
> That's too bad. Thanks for trying it out.
>
> This issue is a race condition, and rr inherently limits the process to
> one core. Can you try "rr record --chaos"? This randomizes the thread
> switching, and is often effective and triggering these kinds of failures
> under rr.
>
> Did not have much luck with --chaos either, the assertion does not trigger.

Hello everyone,

I decided to dig a bit deeper into this issue. I found that the phtreads
mutexes in libx11 were not being initialized by libc, but they were then
used in syscalls. Rebuilding with -pthread fixed the issue, but it
should also not be necessary to do so.

This led me to believe that the hangs/crashes are caused by libc6. I
have updated my system from libc6 2.33-8 to libc6 2.34-1 and I can no
longer reproduce the bug.

@Jurij: What happens if you update your libc6, could you give it a try?
It would be good to get confirmation that the bug was in libc6.

Regards,

--
Göran Weinholt | https://weinholt.se/
Debian Developer | 73 de SA6CJK

[Jurij Smakov]

On Mon, Aug 8, 2022 at 9:50 AM Göran Weinholt <go...@weinholt.se> wrote:

@Jurij: What happens if you update your libc6, could you give it a try?
It would be good to get confirmation that the bug was in libc6.

I can confirm that upgrading libc6 (in my case, to Debian's 2.34-3 package) fixes the problem.

[Göran Weinholt]

Jurij Smakov <ju...@wooyd.org> writes:

> On Mon, Aug 8, 2022 at 9:50 AM Göran Weinholt <go...@weinholt.se> wrote:
>
> @Jurij: What happens if you update your libc6, could you give it a try?
> It would be good to get confirmation that the bug was in libc6.
>
> I can confirm that upgrading libc6 (in my case, to Debian's 2.34-3 package) fixes the problem.

Thank you very much. There is the possibility that libc6 2.34 just hides
the problem. Adam Jackson wrote:

| glibc 2.34 merged several libraries into libc.so.6, including
| libpthread, which would probably explain that.

<https://gitlab.freedesktop.org/xorg/lib/libx11/-/issues/157#note_1500695>

I have a suggested fix for libX11 here, which adds -pthread to the
compilation and linking commands:

<https://gitlab.freedesktop.org/weinholt/libx11/-/commit/a8d4cbc3a46e625c6c4fb4120d2fff56eebcf91c>

It seems harmless, but I've tested it very little.

The discussion is complicated a bit by the fact that there are clients
with genuine bugs (calls to libX11 during event processing), and other
clients that break just because they themselves don't build with
-pthread.

Either way, notion (and glxgears, etc) stopped working because Debian sid
for a short while had glibc 2.33 and a libx11 that called XInitThreads()
without using -pthread, resulting in pthread mutexes being used when
uninitialized.

[Dima Kogan]

Thanks, Göran and Jurij. So are we done? If we're confident, I'll upload
a new notion with the relevant Depends: and close this bug.

Thanks for your work on this!

[Göran Weinholt]

Dima Kogan <dko...@debian.org> writes:

> Thanks, Göran and Jurij. So are we done? If we're confident, I'll upload
> a new notion with the relevant Depends: and close this bug.

Well, pthreads needs to be pulled in some way and then things will be
fine.

Were you planning on putting a later libc6 in Depends:? I hope you're
aware that libc6 is named differently on some archs. If it was me then I
would not want the maintenance burden of that solution. It also seems
like notion should not need to be fixed because the bug is not in
notion.

Personally I think libx11 should be built with -pthread since it is
where the use of pthreads comes from. That would fix this problem for
the other affected X clients.

You could also reassign this bug to libx11-6 and close it. The bug is
really in libx11 1.8.1-1 and is worked around in libx11 1.8.1-2. If a
later libx11 enables the threading stuff again then it should also take
care to build with -pthread so that these problems don't reappear.

Best regards,

[Dima Kogan]

Thanks for the notes, Göran. I was mostly asking to confirm that you are
done investigating, and I should then do something to close this bug
(what, exactly, I haven't thought about very deeply yet). I'll look when
I get the chance, probably in a few weeks.