MCSmear throwing segfault

[Amy Schertz]

Hi all,

I'm trying to generate some MC with MCWrapper v2.10.0, and it seems to
work fine until I get to MCSmear, at which point it throws a segfault.
Has anyone else seen this problem? I've attached the logfile, and pasted
the error output below.

Amy

[aschertz@ifarm2402 omegaDeltaTest]$ gluex_MC.py MC.config 51600 100
batch=0 > log.txt
ls: cannot access
'root://sci-xrootd.jlab.org//osgpool/halld//random_triggers/recon-2018_08-ver02.2/run051600_random.hddm':
No such file or directory
ls: cannot access
'root://nod25.phys.uconn.edu/Gluex/rawdata//random_triggers/recon-2018_08-ver02.2/run051600_random.hddm':
No such file or directory
Warning from GlueXDetectorConstruction::ConstructSDandField -
unsupported sensitive volume TAC1 found in geometry definition.
G4WT0 > Warning from GlueXDetectorConstruction::ConstructSDandField -
unsupported sensitive volume TAC1 found in geometry definition.
JANA ERROR>> didn't sleep full 0.5 seconds!
===========================================================
There was a crash.
This is the entire stack trace of all threads:
===========================================================
Thread 4 (Thread 0x7f0b9cd8e640 (LWP 1377228) "mcsmear"):
#0  0x00007f0bc02d8a3f in wait4 () from /lib64/libc.so.6
#1  0x00007f0bc024b243 in do_system () from /lib64/libc.so.6
#2  0x00007f0bc35ac72c in TUnixSystem::StackTrace() () from
/group/halld/Software/builds/Linux_Alma9-x86_64-gcc11.4.1/root/root-6.24.04/lib/libCore.so
#3  0x00007f0bc35a9d65 in TUnixSystem::DispatchSignals(ESignals) () from
/group/halld/Software/builds/Linux_Alma9-x86_64-gcc11.4.1/root/root-6.24.04/lib/libCore.so
#4  <signal handler called>
#5  0x0000000000000001 in ?? ()
#6  0x00000000014b46ab in
jana::JCalibrationCCDB::GetComment[abi:cxx11]() (this=<optimized out>)
at src/JANA/JCalibrationCCDB.h:86
#7  0x00000000015138a1 in jana::JApplication::GetJCalibration
(this=0x7ffc540a9710, run_number=run_number
entry=51600) at src/JANA/JApplication.cc:1283
#8  0x0000000000803eb6 in MyProcessor::brun (this=0x7ffc540a9690,
loop=0x7f0b98000b60, locRunNumber=51600) at
programs/Simulation/mcsmear/MyProcessor.cc:186
#9  0x0000000001538cc4 in jana::JEventLoop::OneEvent
(this=0x7f0b98000b60) at src/JANA/JEventLoop.cc:677
#10 0x0000000001539314 in jana::JEventLoop::Loop (this=this
entry=0x7f0b98000b60) at src/JANA/JEventLoop.cc:496
#11 0x000000000150e2e5 in LaunchThread (arg=0x7ffc540a9710) at
src/JANA/JApplication.cc:1382
#12 0x00007f0bc0289c02 in start_thread () from /lib64/libc.so.6
#13 0x00007f0bc030ec40 in clone3 () from /lib64/libc.so.6
Thread 3 (Thread 0x7f0ba8b9a640 (LWP 1377208) "mcsmear"):
#0  0x00007f0bc028679a in __futex_abstimed_wait_common () from
/lib64/libc.so.6
#1  0x00007f0bc0288fa0 in pthread_cond_wait
GLIBC_2.3.2 () from /lib64/libc.so.6
#2  0x0000000001520d0b in jana::JApplication::EventBufferThread
(this=0x7ffc540a9710) at src/JANA/JApplication.cc:742
#3  0x000000000152100a in LaunchEventBufferThread (arg=0x7ffc540a9710)
at src/JANA/JApplication.cc:666
#4  0x00007f0bc0289c02 in start_thread () from /lib64/libc.so.6
#5  0x00007f0bc030ec40 in clone3 () from /lib64/libc.so.6
Thread 2 (Thread 0x7f0bbbfff640 (LWP 1377079) "mcsmear"):
#0  0x00007f0bc028679a in __futex_abstimed_wait_common () from
/lib64/libc.so.6
#1  0x00007f0bc0291c48 in __new_sem_wait_slow64.constprop.0 () from
/lib64/libc.so.6
#2  0x00007f0bc0d10680 in XrdPosixFile::DelayedDestroy(void*) () from
/lib64/libXrdPosix.so.3
#3  0x00007f0bbd70e0bc in XrdSysThread_Xeq () from /lib64/libXrdUtils.so.3
#4  0x00007f0bc0289c02 in start_thread () from /lib64/libc.so.6
#5  0x00007f0bc030ec40 in clone3 () from /lib64/libc.so.6
Thread 1 (Thread 0x7f0bbcac8740 (LWP 1377069) "mcsmear"):
#0  0x00007f0bc02d4075 in clock_nanosleep
GLIBC_2.2.5 () from /lib64/libc.so.6
#1  0x00007f0bc02d8c87 in nanosleep () from /lib64/libc.so.6
#2  0x000000000151b80f in jana::JApplication::Run (this=this
entry=0x7ffc540a9710, proc=proc
entry=0x7ffc540a9690, Nthreads=<optimized out>, Nthreads
entry=0) at src/JANA/JApplication.cc:1613
#3  0x000000000073f124 in main (narg=<optimized out>, argv=<optimized
out>) at programs/Simulation/mcsmear/mcsmear.cc:75
===========================================================
The lines below might hint at the cause of the crash.
You may get help by asking at the ROOT forum https://root.cern.ch/forum
Only if you are really convinced it is a bug in ROOT then please submit a
report at https://root.cern.ch/bugs Please post the ENTIRE stack trace
from above as an attachment in addition to anything else
that might help us fixing this issue.
===========================================================
#5  0x0000000000000001 in ?? ()
#6  0x00000000014b46ab in
jana::JCalibrationCCDB::GetComment[abi:cxx11]() (this=<optimized out>)
at src/JANA/JCalibrationCCDB.h:86
#7  0x00000000015138a1 in jana::JApplication::GetJCalibration
(this=0x7ffc540a9710, run_number=run_number
entry=51600) at src/JANA/JApplication.cc:1283
#8  0x0000000000803eb6 in MyProcessor::brun (this=0x7ffc540a9690,
loop=0x7f0b98000b60, locRunNumber=51600) at
programs/Simulation/mcsmear/MyProcessor.cc:186
#9  0x0000000001538cc4 in jana::JEventLoop::OneEvent
(this=0x7f0b98000b60) at src/JANA/JEventLoop.cc:677
#10 0x0000000001539314 in jana::JEventLoop::Loop (this=this
entry=0x7f0b98000b60) at src/JANA/JEventLoop.cc:496
#11 0x000000000150e2e5 in LaunchThread (arg=0x7ffc540a9710) at
src/JANA/JApplication.cc:1382
#12 0x00007f0bc0289c02 in start_thread () from /lib64/libc.so.6
#13 0x00007f0bc030ec40 in clone3 () from /lib64/libc.so.6
===========================================================
JANA ERROR>> didn't sleep full 0.5 seconds!

[Sean Dobbs]

Hi Amy,

Could you please point us to the configuration files you used to run
these MC, like the MC Wrapper config file and the software version set
used.

Cheers,
Sean

> --
> You received this message because you are subscribed to the Google Groups "GlueX Software Help" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to gluex-softwar...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/gluex-software/388ac07c-7cfb-49ba-836d-aeb7612ce559%40jlab.org.

[Amy Schertz]

Hi Sean,

I knew there was something I forgot to add. Can you access the files in
/work/halld2/home/aschertz/myanalysis/batch/simulation/omegaDeltaTest?

My version set is /work/halld2/home/aschertz/builds/version_oct2024.xml.
Let me know if you have trouble accessing any of these files and I'll
try attaching them instead.

Amy

>> You may get help by asking at the ROOT forum https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_forum&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=P1EkFOAcDdzTvaCnPjnJbq9y08ZpEjDJIsODT0l8pqI&m=9ejEN1p1pw3xvMJUVL4dlhSeuxjeLGNIadG3p5M0Uj-U3s9gJj6brBQeY9daQt-n&s=lOKvheRjHW0ph_HxxkF382xyRmllViACkPMIQoDaQdQ&e=

>> Only if you are really convinced it is a bug in ROOT then please submit a

>> report at https://urldefense.proofpoint.com/v2/url?u=https-3A__root.cern.ch_bugs&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=P1EkFOAcDdzTvaCnPjnJbq9y08ZpEjDJIsODT0l8pqI&m=9ejEN1p1pw3xvMJUVL4dlhSeuxjeLGNIadG3p5M0Uj-U3s9gJj6brBQeY9daQt-n&s=XsZO-ukbuPr9rkVGwEFab2Ky0PQHG1WWck7dDFSPRNo&e= Please post the ENTIRE stack trace

>> To view this discussion visit https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msgid_gluex-2Dsoftware_388ac07c-2D7cfb-2D49ba-2D836d-2Daeb7612ce559-2540jlab.org&d=DwIFaQ&c=CJqEzB1piLOyyvZjb8YUQw&r=P1EkFOAcDdzTvaCnPjnJbq9y08ZpEjDJIsODT0l8pqI&m=9ejEN1p1pw3xvMJUVL4dlhSeuxjeLGNIadG3p5M0Uj-U3s9gJj6brBQeY9daQt-n&s=kkkjizB2g2rN0-RBbLmOzaqbigB65PEVucCVRBCLRVs&e= .

[Sean Dobbs]

Yes, I can see them thanks!

> To view this discussion visit https://groups.google.com/d/msgid/gluex-software/95708dd7-4ba3-4c15-b81b-0ab6d51bfd79%40jlab.org.

[Sean Dobbs]

Hi Amy,

I see you are using a local build of halld_sim - maybe you need to update this?
I tried using the latest version.xml, and it did not crash.

Cheers,
Sean