bareos-dir crash when the job is cancelled
23 views
Skip to first unread message
Grigory Trenin
Jul 16, 2025, 4:02:13 PMJul 16
to bareos-users
Hi All,
Just suffered 3 bareos-dir crashes within a week.
$ bareos-dir --version
24.0.2~pre41.988426cda
Backtrace is below.
A null pointer dereference occurred inside glibc at pthread_kill.c:50.
24.0.2~pre41.988426cda
Backtrace is below.
A null pointer dereference occurred inside glibc at pthread_kill.c:50.
Looks like bareos-dir passed an invalid thread_id to pthread_kill().
From the backtrace it looks like it crashed when the job was cancelled.
I have browsed Bareos Git commit log and can see the following very interesting commit message: "...if you try to cancel a job while that job is in the process of ending, then you risk an unfixable crash."
But this is just my guess. Could you please have a look at the backtrace below and advise if that crash is indeed caused by this "unfixable" bug or is it a different issue that should be investigated? This commit mentions that these crashes are unlikely, but I have already seen three ones within a week.
From the backtrace it looks like it crashed when the job was cancelled.
I have browsed Bareos Git commit log and can see the following very interesting commit message: "...if you try to cancel a job while that job is in the process of ending, then you risk an unfixable crash."
But this is just my guess. Could you please have a look at the backtrace below and advise if that crash is indeed caused by this "unfixable" bug or is it a different issue that should be investigated? This commit mentions that these crashes are unlikely, but I have already seen three ones within a week.
(gdb) bt
#0 0x00007fd29888b9b3 in __pthread_kill_implementation (threadid=140541438182976, signo=12, no_tid=0) at pthread_kill.c:50
#1 0x00007fd298f540de in JobControlRecord::MyThreadSendSignal (this=this@entry=0x56047acc47f0, sig=sig@entry=12)
at ../../../../../../../bareos_PR-2177/BUILD/bareos/cmake-build/core/src/lib/../../../../core/src/lib/jcr.cc:440
#2 0x00005604799b4d1e in directordaemon::CancelFileDaemonJob (jcr=0x56047acc47f0, ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/fd_cmds.cc:988
#3 directordaemon::CancelJob(directordaemon::UaContext*, JobControlRecord*) [clone .isra.0] (ua=0x7fd2480160a0, jcr=0x56047acc47f0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/job.cc:696
#4 0x0000560479944dbd in directordaemon::CancelJobs (ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:791
#5 directordaemon::CancelCmd (ua=0x7fd2480160a0, cmd=<optimized out>)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:810
#6 0x000056047994bfaf in directordaemon::UaContext::execute (cmdstruct=<optimized out>, this=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:534
#7 directordaemon::Do_a_command (ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:585
#8 0x000056047993a242 in directordaemon::HandleUserAgentClientRequest (user_agent_socket=0x7fd290000b60)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_server.cc:106
#9 directordaemon::HandleConnectionRequest (config=<optimized out>, arg=0x7fd290000b60)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/socket_server.cc:116
#10 0x00007fd298f6a7d2 in std::function<void* (ConfigurationParser*, void*)>::operator()(ConfigurationParser*, void*) const (__args#1=0x7fd290000b60,
__args#0=<optimized out>, this=0x7fd290001ff8) at /usr/include/c++/11/bits/std_function.h:590
#11 WorkerThread (l=..., ThreadInvokedHandler=..., config=0x56047ab96160, data=0x7fd290000b60, run_condition=...)
at ../../../../../../../bareos_PR-2177/BUILD/bareos/cmake-build/core/src/lib/../../../../core/src/lib/thread_list.cc:201
#12 0x00007fd298f64375 in std::__invoke_impl<void, void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> >(std::__invoke_other, void (*&&)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>&&, std::function<void* (ConfigurationParser*, void*)>&&, ConfigurationParser*&&, void*&&, std::shared_ptr<IsRunningCondition>&&) (
__f=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:61
#13 std::__invoke<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> >(void (*&&)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>&&, std::function<void* (ConfigurationParser*--Type <RET> for more, q to quit, c to continue without paging--c
, void*)>&&, ConfigurationParser*&&, void*&&, std::shared_ptr<IsRunningCondition>&&) (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#14 std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul, 5ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul, 5ul>) (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#15 std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > >::operator()() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > > >::_M_run() (this=<optimized out>)
at /usr/include/c++/11/bits/std_thread.h:211
#17 0x00007fd298cdbad4 in execute_native_thread_routine () from /lib64/libstdc++.so.6
#18 0x00007fd298889d22 in start_thread (arg=<optimized out>) at pthread_create.c:443
#19 0x00007fd29890ed40 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
#0 0x00007fd29888b9b3 in __pthread_kill_implementation (threadid=140541438182976, signo=12, no_tid=0) at pthread_kill.c:50
#1 0x00007fd298f540de in JobControlRecord::MyThreadSendSignal (this=this@entry=0x56047acc47f0, sig=sig@entry=12)
at ../../../../../../../bareos_PR-2177/BUILD/bareos/cmake-build/core/src/lib/../../../../core/src/lib/jcr.cc:440
#2 0x00005604799b4d1e in directordaemon::CancelFileDaemonJob (jcr=0x56047acc47f0, ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/fd_cmds.cc:988
#3 directordaemon::CancelJob(directordaemon::UaContext*, JobControlRecord*) [clone .isra.0] (ua=0x7fd2480160a0, jcr=0x56047acc47f0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/job.cc:696
#4 0x0000560479944dbd in directordaemon::CancelJobs (ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:791
#5 directordaemon::CancelCmd (ua=0x7fd2480160a0, cmd=<optimized out>)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:810
#6 0x000056047994bfaf in directordaemon::UaContext::execute (cmdstruct=<optimized out>, this=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:534
#7 directordaemon::Do_a_command (ua=0x7fd2480160a0)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_cmds.cc:585
#8 0x000056047993a242 in directordaemon::HandleUserAgentClientRequest (user_agent_socket=0x7fd290000b60)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/ua_server.cc:106
#9 directordaemon::HandleConnectionRequest (config=<optimized out>, arg=0x7fd290000b60)
at ../../../../../../../bareos-testing_bareos-24/BUILD/bareos/cmake-build/core/src/dird/../../../../core/src/dird/socket_server.cc:116
#10 0x00007fd298f6a7d2 in std::function<void* (ConfigurationParser*, void*)>::operator()(ConfigurationParser*, void*) const (__args#1=0x7fd290000b60,
__args#0=<optimized out>, this=0x7fd290001ff8) at /usr/include/c++/11/bits/std_function.h:590
#11 WorkerThread (l=..., ThreadInvokedHandler=..., config=0x56047ab96160, data=0x7fd290000b60, run_condition=...)
at ../../../../../../../bareos_PR-2177/BUILD/bareos/cmake-build/core/src/lib/../../../../core/src/lib/thread_list.cc:201
#12 0x00007fd298f64375 in std::__invoke_impl<void, void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> >(std::__invoke_other, void (*&&)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>&&, std::function<void* (ConfigurationParser*, void*)>&&, ConfigurationParser*&&, void*&&, std::shared_ptr<IsRunningCondition>&&) (
__f=<optimized out>, __f=<optimized out>) at /usr/include/c++/11/bits/invoke.h:61
#13 std::__invoke<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> >(void (*&&)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>&&, std::function<void* (ConfigurationParser*--Type <RET> for more, q to quit, c to continue without paging--c
, void*)>&&, ConfigurationParser*&&, void*&&, std::shared_ptr<IsRunningCondition>&&) (__fn=<optimized out>) at /usr/include/c++/11/bits/invoke.h:96
#14 std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > >::_M_invoke<0ul, 1ul, 2ul, 3ul, 4ul, 5ul>(std::_Index_tuple<0ul, 1ul, 2ul, 3ul, 4ul, 5ul>) (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:259
#15 std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > >::operator()() (this=<optimized out>) at /usr/include/c++/11/bits/std_thread.h:266
#16 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)> const&, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition>), std::shared_ptr<ThreadListContainer>, std::function<void* (ConfigurationParser*, void*)>, ConfigurationParser*, void*, std::shared_ptr<IsRunningCondition> > > >::_M_run() (this=<optimized out>)
at /usr/include/c++/11/bits/std_thread.h:211
#17 0x00007fd298cdbad4 in execute_native_thread_routine () from /lib64/libstdc++.so.6
#18 0x00007fd298889d22 in start_thread (arg=<optimized out>) at pthread_create.c:443
#19 0x00007fd29890ed40 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
Kind regard,
Grigory Trenin
Reply all
Reply to author
Forward
0 new messages